PuSH - Publication Server of Helmholtz Zentrum München

Möllers, A.* ; Immer, A.* ; Fortuin, V. ; Isufi, E.*

Hodge-aware contrastive learning.

In: (49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024, 14-19 April 2024, Seoul). 345 E 47th St, New York, Ny 10017 Usa: Ieee, 2024. 9746-9750 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings)
DOI
Simplicial complexes prove effective in modeling data with multiway dependencies, such as data defined along the edges of networks or within other higher-order structures. Their spectrum can be decomposed into three interpretable subspaces via the Hodge decomposition, resulting foundational in numerous applications. We leverage this decomposition to develop a contrastive self-supervised learning approach for processing simplicial data and generating embeddings that encapsulate specific spectral information. Specifically, we encode the pertinent data invariances through simplicial neural networks and devise augmentations that yield positive contrastive examples with suitable spectral properties for downstream tasks. Additionally, we reweight the significance of negative examples in the contrastive loss, considering the similarity of their Hodge components to the anchor. By encouraging a stronger separation among less similar instances, we obtain an embedding space that reflects the spectral properties of the data. The numerical results on two standard edge flow classification tasks show a superior performance even when compared to supervised learning techniques. Our findings underscore the importance of adopting a spectral perspective for contrastive learning with higher-order data.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Keywords Contrastive Learning ; Hodge Laplacian ; Simplicial Filter
ISSN (print) / ISBN 1520-6149
Conference Title 49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
Conference Date 14-19 April 2024
Conference Location Seoul
Quellenangaben Volume: , Issue: , Pages: 9746-9750 Article Number: , Supplement: ,
Publisher Ieee
Publishing Place 345 E 47th St, New York, Ny 10017 Usa
Institute(s) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Grants BrancoWeiss Fellowship
Max Planck ETH Center for Learning Systems doctoral fellowship
TU Delft AI Labs Programme