PuSH - Publication Server of Helmholtz Zentrum München

Reinke, A.* ; Tizabi, M.D.* ; Baumgartner, M.* ; Eisenmann, M.* ; Heckmann-Noetzel, D.* ; Kavur, A.E.* ; Raedsch, T.* ; Sudre, C.H.* ; Acion, L.* ; Antonelli, M.* ; Arbel, T.* ; Bakas, S.* ; Benis, A.* ; Buettner, F.* ; Cardoso, M.J.* ; Cheplygina, V.* ; Chen, J.* ; Christodoulou, E.* ; Cimini, B.A.* ; Farahani, K.* ; Ferrer, L.* ; Galdran, A.* ; Van Ginneken, B.* ; Glocker, B.* ; Godau, P.* ; Hashimoto, D.A.* ; Hoffman, M.M.* ; Huisman, M.* ; Isensee, F.* ; Jannin, P.* ; Kahn, C.E.* ; Kainmueller, D.* ; Kainz, B.* ; Karargyris, A.* ; Kleesiek, J.* ; Kofler, F. ; Kooi, T.* ; Kopp-Schneider, A.* ; Kozubek, M.* ; Kreshuk, A.* ; Kurc, T.* ; Landman, B.A.* ; Litjens, G.* ; Madani, A.* ; Maier-Hein, K.* ; Martel, A.L.* ; Meijering, E.* ; Menze, B.* ; Moons, K.G.M.* ; Mueller, H.* ; Nichyporuk, B.* ; Nickel, F.* ; Petersen, J.* ; Rafelski, S.M.* ; Rajpoot, N.* ; Reyes, M.* ; Riegler, M.A.* ; Rieke, N.* ; Saez-Rodriguez, J.* ; Sánchez, C.I.* ; Shetty, S.* ; Summers, R.M.* ; Taha, A.A.* ; Tiulpin, A.* ; Tsaftaris, S.A.* ; van Calster, B.* ; Varoquaux, G.* ; Yaniv, Z.R.* ; Jaeger, P.F.* ; Maier-Hein, L.*

Understanding metric-related pitfalls in image analysis validation.

Nat. Methods 21, 182–194 (2024)
DOI
Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation. This Perspective presents a reliable and comprehensive source of information on pitfalls related to validation metrics in image analysis, with an emphasis on biomedical imaging.
Altmetric
Additional Metrics?
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Corresponding Author
Keywords Segmentation
ISSN (print) / ISBN 1548-7091
e-ISSN 1548-7105
Journal Nature Methods
Quellenangaben Volume: 21, Issue: , Pages: 182–194 Article Number: , Supplement: ,
Publisher Nature Publishing Group
Publishing Place New York, NY
Non-patent literature Publications
Reviewing status Peer reviewed
Grants Research Chairs and Senior Research Fellowships scheme
natural sciences and engineering research council of canada
NIH
Wellcome/EPSRC Centre for Medical Engineering
National Institute of Neurological Disorders and Stroke (NINDS) of the National Institutes of Health (NIH)
Intramural Research Program of the National Institutes of Health Clinical Center
Surgical Oncology Program of the National Center for Tumor Diseases (NCT) Heidelberg
European Research Council (ERC) under the European Union
Helmholtz Association of German Research Centers in the scope of the Helmholtz Imaging Incubator
Ministry of Education, Youth and Sports of the Czech Republic
National Institutes of Health
Canon Medical
Terttu foundation
Finnish Foundation for Cardiovascular Research, Wellbeing Services County of North Ostrobothnia
Academy of Finland
Alzheimer's Society Junior Fellowship
European Research Council
Dutch Cancer Association
Dutch Research Council
Helmholtz Association of German Research Centers in the scope of the Helmholtz Imaging Incubator (HI)