Clues to quality of journals
Open Access Policy of Helmholtz Association 2016
CC Licencences
Publication Metrics
Home
Deutsch
New EVA Application
Advanced Search
Browse by ...
Publication overview
Statistics (last 5 years)
OA Publications
Submit Publication
Import publication from...
Report missing publication
Contact persons
Help
Text
Endnote (RIS)
BibTeX
Publ. Version/Full Text
DOI
 |
Open Access Hybrid |
|
as soon as is submitted to ZB.
Labelling instructions matter in biomedical image analysis.
Nat. Mach. Intell. 5, 273–283 (2023)
Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labelling instructions are key. Despite their importance, their optimization remains largely unexplored. Here we present a systematic study of labelling instructions and their impact on annotation quality in the field. Through comprehensive examination of professional practice and international competitions registered at the Medical Image Computing and Computer Assisted Intervention Society, the largest international society in the biomedical imaging field, we uncovered a discrepancy between annotators’ needs for labelling instructions and their current quality and availability. On the basis of an analysis of 14,040 images annotated by 156 annotators from four professional annotation companies and 708 Amazon Mechanical Turk crowdworkers using instructions with different information density levels, we further found that including exemplary images substantially boosts annotation performance compared with text-only descriptions, while solely extending text descriptions does not. Finally, professional annotators constantly outperform Amazon Mechanical Turk crowdworkers. Our study raises awareness for the need of quality standards in biomedical image analysis labelling instructions.
Impact Factor
Scopus SNIP
Web of Science
Times Cited
Times Cited
Altmetric
23.800
0.000
8
Annotations
Special Publikation
Hide on homepage
Publication type
Article: Journal article
Document type
Scientific Article
Keywords
Quality
Language
english
Publication Year
2023
HGF-reported in Year
2023
ISSN (print) / ISBN
2522-5839
e-ISSN
2522-5839
Journal
Nature machine intelligence
Quellenangaben
Volume: 5,
Issue: 3,
Pages: 273–283
Publisher
Springer
Publishing Place
[London]
Reviewing status
Peer reviewed
Institute(s)
Helmholtz Pioneer Campus (HPC)
POF-Topic(s)
30205 - Bioengineering and Digital Health
Research field(s)
Pioneer Campus
PSP Element(s)
G-510001-001
Grants
Deutsches Krebsforschungszentrum (DKFZ)
WOS ID
000942663300001
Scopus ID
85149140450
Erfassungsdatum
2023-11-29