PuSH - Publication Server of Helmholtz Zentrum München

Hasny, M. ; di Folco, M. ; Bressem, K.* ; Schnabel, J.A.

Tables Guide Vision: Learning to See the Heart through Tabular Data.

In: (2026 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2026, 6-10 March 2026, Tucson). 2026. 1716-1725 (Proceedings 2026 IEEE Cvf Winter Conference on Applications of Computer Vision Wacv 2026)
DOI
Contrastive learning methods in computer vision typically rely on augmented views of the same image or multimodal pretraining strategies that align paired modalities. However, these approaches often overlook semantic relationships between distinct instances, leading to false negatives when semantically similar samples are treated as negatives. This limitation is especially critical in medical imaging domains such as cardiology, where demographic and clinical attributes play a critical role in assessing disease risk and patient outcomes. We introduce a tabular-guided contrastive learning framework that leverages clinically relevant tabular data to identify patient-level similarities and construct more meaningful pairs, enabling semantically aligned representation learning without requiring joint embeddings across modalities. Additionally, we adapt the k-NN algorithm for zero-shot prediction to overcome the lack of zero-shot capability in unimodal representations. We demonstrate the strength of our methods using a large cohort of short-axis cardiac MR images and clinical attributes, where tabular data helps to more effectively distinguish between patient subgroups. Evaluation on downstream tasks, including fine-tuning, linear probing, and zero-shot prediction of cardiovascular artery diseases and cardiac phenotypes, shows that incorporating tabular data guidance yields stronger visual representations than conventional methods that rely solely on image augmentation or combined image-tabular embeddings. Further, we show that our method can generalize to natural images by evaluating it on a car advertisement dataset. Code is available at this link.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Keywords Cardiac Mri ; Contrastive Learning ; Visual Representation
ISSN (print) / ISBN [9798331555115]
Conference Title 2026 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2026
Conference Date 6-10 March 2026
Conference Location Tucson
Quellenangaben Volume: , Issue: , Pages: 1716-1725 Article Number: , Supplement: ,
Institute(s) Institute for Machine Learning in Biomed Imaging (IML)