PuSH - Publikationsserver des Helmholtz Zentrums München

Gindra, R. ; Palla, G. ; Nguyen, M. ; Wagner, S. ; Tran, M. ; Theis, F.J. ; Saur, D.* ; Crawford, L.* ; Peng, T.

A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics.

In: (Proceedings 2025 IEEE Cvf International Conference on Computer Vision Workshops Iccv W 2025, 19-20 October 2025, Honolulu). 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa: Ieee Computer Soc, 2025. 1193-1203 (Proceedings 2025 IEEE Cvf International Conference on Computer Vision Workshops Iccv W 2025)
DOI
Spatial transcriptomics enables simultaneous measurement of gene expression and tissue morphology, offering unprecedented insights into cellular organization and disease mechanisms. However, the field lacks comprehensive benchmarks for evaluating multimodal learning methods that leverage both histology images and gene expression data. Here, we present HESCAPE, a large-scale benchmark for cross-modal contrastive pretraining in spatial transcriptomics, built on a curated pan-organ dataset spanning 6 different gene panels and 54 donors. We systematically evaluated state-of-the-art image and gene expression encoders across multiple pre-training strategies and assessed their effectiveness on two downstream tasks: gene mutation classification and gene expression prediction. Our benchmark demonstrates that gene expression encoders are the primary determinant of strong representational alignment, and that gene models pretrained on spatial transcriptomics data outperform both those trained without spatial data and simple baseline approaches. However, downstream task evaluation reveals a striking contradiction: while contrastive pretraining consistently improves gene mutation classification performance, it degrades direct gene expression prediction compared to baseline encoders trained without cross-modal objectives. We identify batch effects as a key factor that interferes with effective cross-modal alignment. Our findings highlight the critical need for batch-robust multimodal learning approaches in spatial transcriptomics. To accelerate progress in this direction, we release HESCAPE, providing standardized datasets, evaluation protocols, and benchmarking tools for the community, at https://github.com/peng-lab/hescape
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Konferenzbeitrag
Schlagwörter Benchmark ; Foundation Models ; Genomics ; Histopathology ; Multimodal Learning ; Spatial-omics; Foundation Model
ISSN (print) / ISBN [9798331589882]
Konferenztitel Proceedings 2025 IEEE Cvf International Conference on Computer Vision Workshops Iccv W 2025
Konferzenzdatum 19-20 October 2025
Konferenzort Honolulu
Quellenangaben Band: , Heft: , Seiten: 1193-1203 Artikelnummer: , Supplement: ,
Verlag Ieee Computer Soc
Verlagsort 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa
Institut(e) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Institute of AI for Health (AIH)
Institute of Computational Biology (ICB)
Förderungen Helmholtz Association's Initiative and Networking Fund on the HAICORE@FZJ partition
de.NBI Cloud within the German Network for Bioinformatics Infrastructure (de.NBI)
Add-on Fellowship of the Joachim Herz Foundation
Helmholtz Association under the joint research school "Munich School for Data Science