PuSH - Publication Server of Helmholtz Zentrum München

Roth, B. ; Koch, V. ; Wagner, S. ; Schnabel, J.A. ; Marr, C. ; Peng, T.

Low-resource finetuning of foundation models beats state-of-the-art in histopathology.

In: (Proceedings - International Symposium on Biomedical Imaging, 27-30 May 2024, Athen). 2024. DOI: 10.1109/ISBI56570.2024.10635695 (Proceedings - International Symposium on Biomedical Imaging)
DOI
To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors with weakly-supervised learning. The performance of this workflow strongly depends on the quality of the extracted features. Recently, foundation models in computer vision showed that leveraging huge amounts of data through supervised or self-supervised learning improves feature quality and generalizability for a variety of tasks. In this study, we benchmark the most popular vision foundation models as feature extractors for histopathology data. We evaluate the models in two settings: slide-level classification and patch-level classification. We show that foundation models are a strong baseline. Our experiments demonstrate that by finetuning a foundation model on a single GPU for only two hours or three days depending on the dataset, we can match or outperform state-of-the-art feature extractors for computational pathology. These findings imply that even with little resources one can finetune a feature extractor tailored towards a specific downstream task and dataset. This is a considerable shift from the current state, where only few institutions with large amounts of resources and datasets are able to train a feature extractor. We publish all code used for training and evaluation as well as the finetuned models1.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Corresponding Author
Keywords Foundation Models ; Histopathology ; Medical Imaging ; Self-supervised Learning
ISSN (print) / ISBN 1945-7928
e-ISSN 1945-8452
Conference Title Proceedings - International Symposium on Biomedical Imaging
Conference Date 27-30 May 2024
Conference Location Athen
Non-patent literature Publications
Institute(s) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Institute of AI for Health (AIH)
Institute for Machine Learning in Biomed Imaging (IML)