PuSH - Publication Server of Helmholtz Zentrum München

Mayr, J.* ; Reithmeir, A. ; di Folco, M. ; Schnabel, J.A.

Covariance Descriptors Meet General Vision Encoders: Riemannian Deep Learning for Medical Image Classification.

In: (23rd IEEE International Symposium on Biomedical Imaging, ISBI 2026, 8-11 April 2026, London). 2026. (Proceedings International Symposium on Biomedical Imaging ; 2026-April)
DOI
Covariance descriptors capture second-order statistics of image features. They have shown strong performance in general computer vision tasks, but remain underexplored in medical imaging. We investigate their effectiveness for both conventional and learning-based medical image classification, with a particular focus on SPDNet, a classification network specifically designed for symmetric positive definite (SPD) matrices. We propose constructing covariance descriptors from features extracted by pre-trained general vision encoders (GVEs) and compare them with handcrafted descriptors. Two GVEs - DINOv2 and MedSAM - are evaluated across eleven binary and multi-class datasets from the MedMNIST benchmark. Our results show that covariance descriptors derived from GVE features consistently outperform those derived from handcrafted features. Moreover, SPDNet yields superior performance to state-of-the-art methods when combined with DINOv2 features. Our findings highlight the potential of combining covariance descriptors with powerful pretrained vision encoders for medical image analysis. Our code is publicly available at https://github.com/compai-lab/2026-isbi-mayr.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Keywords Classification ; Riemannian Deep Learning ; Vision Features
ISSN (print) / ISBN 1945-7928
e-ISSN 1945-8452
Conference Title 23rd IEEE International Symposium on Biomedical Imaging, ISBI 2026
Conference Date 8-11 April 2026
Conference Location London
Quellenangaben Volume: 2026-April Issue: , Pages: , Article Number: , Supplement: ,
Institute(s) Institute for Machine Learning in Biomed Imaging (IML)