PuSH - Publication Server of Helmholtz Zentrum München

Schilling, M.P.* ; Scherr, T.* ; Muenke, F.R.* ; Neumann, O.* ; Schutera, M.* ; Mikut, R.* ; Reischl, M.* ; Schilling, M.*

Automated Annotator Variability Inspection for Biomedical Image Segmentation.

IEEE Access 10, 2753-2765 (2022)
Publ. Version/Full Text DOI
Open Access Gold
Creative Commons Lizenzvertrag
Supervised deep learning approaches for automated diagnosis support require datasets annotated by experts. Intra-annotator variability of a single annotator and inter-annotator variability between annotators can affect the quality of the diagnosis support. As medical experts will always differ in annotation details, quantitative studies concerning the annotation quality are of particular interest. A consistent and noise-free annotation of large-scale datasets by, for example, dermatologists or pathologists is a current challenge. Hence, methods are needed to automatically inspect annotations in datasets. In this paper, we categorize annotation noise in image segmentation tasks, present methods to simulate annotation noise, and examine the impact on the segmentation quality. Two novel automated methods to identify intra-annotator and inter-annotator inconsistencies based on uncertainty-aware deep neural networks are proposed. We demonstrate the benefits of our automated inspection methods such as focused re-inspection of noisy annotations or the detection of generally different annotation styles using the biomedical ISIC 2017 Melanoma image segmentation dataset.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Keywords Annotations ; Image Segmentation ; Task Analysis ; Noise Measurement ; Uncertainty ; Inspection ; Training ; Artificial Neural Networks ; Automation ; Machine Learning ; Segmentation ; Image Processing
ISSN (print) / ISBN 2169-3536
e-ISSN 2169-3536
Journal IEEE Access
Quellenangaben Volume: 10, Issue: , Pages: 2753-2765 Article Number: , Supplement: ,
Publisher IEEE
Publishing Place 445 Hoes Lane, Piscataway, Nj 08855-4141 Usa
Reviewing status Peer reviewed
Institute(s) Helmholtz AI - KIT (HAI - KIT)
Grants KIT-Publication Fund of the Karlsruhe Institute of Technology
Helmholtz Imaging Platform
Helmholtz Association's Initiative and Networking Fund through the Helmholtz AI
Federal Ministry of Education and Research
HoreKa Supercomputer through the Ministry of Science, Research, and the Arts Baden-Wurttemberg
KIT Future Fields Project "Screening Platform for Personalized Oncology (SPPO)''