PuSH - Publikationsserver des Helmholtz Zentrums München

Watter, M.* ; Engel, F.* ; Kalantari, A.* ; Giuliani, C.* ; Schuller, K. ; Franzke, C.* ; Sperandio, M.* ; Binder, H.* ; Kaier, K.*

Accelerating metadata annotation in collaborative research centers: A hybrid AI workflow for biomedical entities.

Bioinfo. Adv. 6:vbag283 (2026)
Verlagsversion Postprint DOI
Open Access Gold
Creative Commons Lizenzvertrag
Background: Collaborative Research Centers rely on FAIR-compliant, richly structured metadata, yetmanual annotation is a major bottleneck. We implemented a search-augmented large languagemodel (LLM) workflow within a local research data management system to pre-annotate biomedicalentities, using human-in-the-loop verification to ensure data quality.Methods: The pipeline uses Gemini 3 Pro in a two-step prompting strategy: (1) identify datasetdeposits and stable identifiers in articles converted to Markdown; (2) extract structured fields fromcurated repository landing pages rendered via a headless browser. To handle a highly hierarchicalmetadata schema, we flattened the schema for prompting and remapped outputs to strict JSON withgranular provenance tags. Authors received pre-filled metadata and could accept, edit, or deleteentries (TP, FP, FN mapping). Performance metrics (precision, recall, F1) were estimated asproportions and synthesized via random-effects meta-analysis.Results: Among 51 screened articles, the LLM identified a repository deposit in 31; authorsresponded for 17 (55% of articles with an identified deposit; 33% of all screened articles), yielding 39verified datasets. False positives were rare (mean 0.23) and false negatives low (mean 1.46).Precision was consistently high at 99.65% (95% CI 98.42%–100.00%). Recall showed morevariability at 93.75% (95% CI 89.79%–96.96%), and the F1 score reached 96.17% (95% CI 93.78%–98.11%).Conclusions: The hybrid workflow achieved high author-verified precision with moderate recall, shiftingeffort from drafting to reviewing while maintaining schema compliance. As these metrics rely on authoracceptance and response rates were modest, broader, multi-site validation is needed.
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Journalartikel
Dokumenttyp Wissenschaftlicher Artikel
Schlagwörter Metadata ; Workflow ; Annotation ; Identifier ; False Positive Paradox ; Schema (genetic Algorithms) ; Json ; False Positives And False Negatives
ISSN (print) / ISBN 2635-0041
e-ISSN 2635-0041
Zeitschrift Bioinformatics Advances
Quellenangaben Band: 6, Heft: 1, Seiten: , Artikelnummer: vbag283 Supplement: ,
Verlag Oxford University Press
Begutachtungsstatus Peer reviewed
Institut(e) Strategy and Digitalization (DIG)