PuSH - Publication Server of Helmholtz Zentrum München

Albert, T.* ; Eskofier, B.M. ; Zanca, D.*

From patches to objects: Exploiting spatial reasoning for better visual representations.

SN appl. sci. 6:232 (2024)
Publ. Version/Full Text DOI
Open Access Gold
Creative Commons Lizenzvertrag
As the field of deep learning steadily transitions from the realm of academic research to practical application, the significance of self-supervised pretraining methods has become increasingly prominent. These methods, particularly in the image domain, offer a compelling strategy to effectively utilize the abundance of unlabeled image data, thereby enhancing downstream tasks’ performance. In this paper, we propose Spatial Reasoning, a novel auxiliary pretraining method that takes advantage of a more flexible formulation of contrastive learning by introducing spatial reasoning as an auxiliary task for discriminative self-supervised methods. Spatial Reasoning works by having the network predict the relative distances between sampled non-overlapping patches. We argue that this forces the network to learn more detailed and intricate internal representations of the objects and the relationships between their constituting parts. Our experiments demonstrate substantial improvement in downstream performance in linear evaluation compared to similar work and provide directions for further research into spatial reasoning.
Altmetric
Additional Metrics?
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Corresponding Author
Keywords Contrastive Learning ; Linear Evaluation ; Relational Reasoning ; Self-supervised Learning ; Spatial Reasoning
ISSN (print) / ISBN 2523-3963
e-ISSN 2523-3971
Quellenangaben Volume: 6, Issue: 5, Pages: , Article Number: 232 Supplement: ,
Publisher Springer
Publishing Place [Cham]
Non-patent literature Publications
Reviewing status Peer reviewed
Institute(s) Institute of AI for Health (AIH)
Grants Projekt DEAL