PuSH - Publication Server of Helmholtz Zentrum München

Kofler, F. ; Wahle, J.* ; Ezhov, I.* ; Wagner, S. ; Al-Maskari, R. ; Gryska, E.* ; Todorov, M.I. ; Bukas, C. ; Meissen, F.* ; Peng, T. ; Ertürk, A. ; Rueckert, D.* ; Heckemann, R.* ; Kirschke, J.* ; Zimmer, C.* ; Wiestler, B.* ; Menze, B.* ; Piraud, M.

Approaching Peak Ground Truth.

In: (Proceedings - International Symposium on Biomedical Imaging, 18-21 April 2023, Cartagena, Colombia). 345 E 47th St, New York, Ny 10017 Usa: Ieee, 2023. 6 (Proceedings - International Symposium on Biomedical Imaging ; 2023-April)
DOI
Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the biomedical domain, annotations are subjective and suffer from low inter-and intra-rater reliability. Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores. Here, the theoretical concept of Peak Ground Truth (PGT) is introduced. PGT marks the point beyond which an increase in similarity with the reference annotation stops translating to better Real World Model Performance (RWMP). Additionally, a quantitative technique to approximate PGT by computing inter- and intra-rater reliability is proposed. Finally, four categories of PGT-aware strategies to evaluate and improve model performance are reviewed.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Corresponding Author
Keywords Annotation ; Deep Learning ; Ground Truth ; Machine Learning ; Reference ; Segmentation; Label Noise; Classification
ISSN (print) / ISBN 1945-7928
e-ISSN 1945-8452
Conference Title Proceedings - International Symposium on Biomedical Imaging
Conference Date 18-21 April 2023
Conference Location Cartagena, Colombia
Quellenangaben Volume: 2023-April, Issue: , Pages: 6 Article Number: , Supplement: ,
Publisher Ieee
Publishing Place 345 E 47th St, New York, Ny 10017 Usa
Non-patent literature Publications
Institute(s) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Institute for Tissue Engineering and Regenerative Medicine (ITERM)
Grants
Helmut Horten Foundation
BMBF
DFG
ERC
Technical University of Munich - Institute for Advanced Study - German Excellence Initiative
Add-on Fellowship of the Joachim Herz Foundation
Helmholtz Association under the joint research school "Munich School for Data Science MUDS"
DComEX
Translational Brain Imaging Training Network (TRABIT) under the European Union's 'Horizon 2020' research & innovation program
Deutsche Forschungsgemeinschaft (DFG) through TUM International Graduate School of Science and Engineering (IGSSE)