PuSH - Publikationsserver des Helmholtz Zentrums München

Engel, L.* ; Bergmann, M.* ; Kammel, C.* ; Ullmann, I.* ; Eskofier, B.M. ; Vossiek, M.*

Fully automated, AI-driven multimodal annotation framework for human-centric radar applied to deep learning-based people localization.

IEEE J. Microw., DOI: 10.1109/JMW.2025.3642570 (2025)
Verlagsversion DOI
Open Access Gold
Creative Commons Lizenzvertrag
Radar-based machine learning pipelines require extensive annotated datasets. However, producing large volumes of precise labels remains prohibitively laborious and prone to inconsistency, as radar signals lack a direct visual correspondence. To address this limitation, we introduce a fully automated, multi-modal annotation pipeline built around our custom RadarBox that co-registers a FMCW MIMO radar with an Azure Kinect RGB-D camera. Precise spatial calibration and hardware-level synchronization yield exact pixel-to-radar alignment. RGB images undergo panoptic segmentation to generate per-pixel human masks, which are fused with depth measurements to reconstruct a voxelized surface mesh. We extract 3D joint positions from the Kinect Body Tracking SDK and apply a bidirectional Kalman filter to derive precise per-joint positions and velocity vectors free from sudden, non-physiological fluctuations. These enhanced labels are projected into 5D radar cube slices and target lists through robust spatio-temporal association. As a demonstration, we train a deep neural network on annotated radar target lists for indoor people localization, achieving a mean positional error of 0.31 m and 91.8% occupancy accuracy, even under occlusion. Unlike prior semi-automatic or heuristic-based methods, our approach delivers consistent 5D labels at scale, bridging spatial, temporal, and Doppler dimensions, and thus paves the way for large-scale, learning-based radar sensing in human-centered applications.
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Journalartikel
Dokumenttyp Wissenschaftlicher Artikel
Schlagwörter Ai-driven ; Automatic Labeling ; Deep Learning ; Human-centric ; People Localization ; Radar
ISSN (print) / ISBN 2692-8388
e-ISSN 2692-8388
Zeitschrift IEEE Journal of Microwaves
Verlag IEEE
Verlagsort 445 Hoes Lane, Piscataway, Nj 08855-4141 Usa
Begutachtungsstatus Peer reviewed
Institut(e) Institute of AI for Health (AIH)
Förderungen Bavarian Ministry of Economic Affairs, Regional Development and Energy
Deutsche Forschungsgemeinschaft (DFG, German Research foundation)