PuSH - Publikationsserver des Helmholtz Zentrums München

Grashei, C.* ; Brechenmacher, C. ; Umer, R.M. ; Liu, J.* ; Marr, C. ; Schüffler, P.* ; Szczurek, E.

Pathryoshka: Compressing Pathology Foundation Models via Multi-teacher Knowledge Distillation with Nested Embeddings.

In: (Computer Vision – ECCV 2026). Berlin [u.a.]: Springer, 2026. 76-94 (Lect. Notes Comput. Sc. ; 17074 LNCS)
DOI
Foundation models (FMs) have driven significant progress in computational pathology. These models can easily exceed a billion parameters and produce high–dimensional embeddings, thus limiting their applicability for research or clinical use when computing resources are tight. Here we introduce Pathryoshka, a novel multi–teacher distillation framework inspired by agglomerative models and Matryoshka representation learning to reduce pathology FM sizes while allowing for adaptable embedding dimensions. We evaluate our framework with a distilled model on ten public pathology benchmarks with varying downstream tasks. Compared to its much larger teachers, Pathryoshka reduces the model size by 86–92% at on–par performance. It outperforms state–of–the–art single–teacher distillation models of comparable size by a median margin of 7.0 and other pathology multi–teacher distillation by 5.3 percentage points in accuracy. By enabling efficient deployment without sacrificing accuracy or representational richness, Pathryoshka democratizes access to state–of–the–art pathology FMs for the broader research and clinical community.
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Konferenzbeitrag
Schlagwörter Computational Pathology ; Foundation Model ; Knowledge Distillation
ISSN (print) / ISBN 0302-9743
e-ISSN 1611-3349
Konferenztitel Computer Vision – ECCV 2026
Zeitschrift Lecture Notes in Computer Science
Quellenangaben Band: 17074 LNCS, Heft: , Seiten: 76-94 Artikelnummer: , Supplement: ,
Verlag Springer
Verlagsort Berlin [u.a.]
Institut(e) Institute of AI for Health (AIH)