PuSH - Publication Server of Helmholtz Zentrum München

Berger, A.H.* ; Lux, L.* ; Shit, S.* ; Ezhov, I.* ; Kaissis, G. ; Menten, M.J.* ; Rueckert, D.* ; Paetzold, J.C.*

Cross-domain and cross-dimension learning for image-to-graph transformers.

In: (2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025, 28 February - 4 March 2025, Tucson). 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa: Ieee Computer Soc, 2025. 64-74 (Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025)
Postprint Research data DOI
Direct image-to-graph transformation is a challenging task that involves solving object detection and relationship prediction in a single model. Due to this task's complexity, large training datasets are rare in many domains, making the training of deep-learning methods challenging. This data sparsity necessitates transfer learning strategies akin to the state-of-the-art in general computer vision. In this work, we introduce a set of methods enabling cross-domain and cross-dimension learning for image-to-graph trans-formers. We propose (1) a regularized edge sampling loss to effectively learn object relations in multiple domains with different numbers of edges, (2) a domain adaptation frame-work for image-to-graph transformers aligning image- and graph-level features from different domains, and (3) a projection function that allows using 2D data for training 3D transformers. We demonstrate our method's utility in cross-domain and cross-dimension experiments, where we utilize labeled data from 2D road networks for simultaneous learning in vastly different target domains. Our method consistently outperforms standard transfer learning and self-supervised pretraining on challenging benchmarks, such as retinal or whole-brain vessel graph extraction.11Code: github.com/AlexanderHBerger/cross-dim_i2g
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Corresponding Author
Keywords Domain Adaptation ; Image-to-graph ; Transfer Learning
ISSN (print) / ISBN [9798331510831]
Conference Title 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025
Conference Date 28 February - 4 March 2025
Conference Location Tucson
Quellenangaben Volume: , Issue: , Pages: 64-74 Article Number: , Supplement: ,
Publisher Ieee Computer Soc
Publishing Place 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa
Non-patent literature Publications
Institute(s) Institute for Machine Learning in Biomed Imaging (IML)
Grants Bavarian Collaborative Research Project PRIPREKI of the Free State of Bavaria Funding Programme "Artificial Intelligence - Data Science" - Konrad Zuse School of Excellence in Reliable AI (relAI)
Medical Informatics Initiative
German Ministry of Education and Research
Stiftung der Deutschen Wirtschaft - German Research Foundation