PuSH - Publication Server of Helmholtz Zentrum München

Keller, M.* ; Arora, V.* ; Dakri, A.* ; Chandhok, S.* ; Machann, J. ; Fritsche, A. ; Black, M.J.* ; Pujades, S.*

HIT: Estimating internal human implicit tissues from the body surface.

In: (IEEE Conference on Computer Vision and Pattern Recognition, 16-22 June 2024, Seattle, WA). 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa: Ieee Computer Soc, 2024. 3480-3490
DOI
The creation of personalized anatomical digital twins is important in the fields of medicine, computer graphics, sports science, and biomechanics. To observe a subject's anatomy, expensive medical devices (MRI or CT) are required and the creation of the digital model is often time-consuming and involves manual effort. Instead, we leverage the fact that the shape of the body surface is correlated with the internal anatomy; e.g. from surface observations alone, one can predict body composition and skeletal structure. In this work, we go further and learn to infer the 3D location of three important anatomic tissues: subcutaneous adipose tissue (fat), lean tissue (muscles and organs), and long bones. To learn to infer these tissues, we tackle several key challenges. We first create a dataset of human tissues by segmenting full-body MRI scans and registering the SMPL body mesh to the body surface. With this dataset, we train HIT (Human Implicit Tissues), an implicit function that, given a point inside a body, predicts its tissue class. HIT leverages the SMPL body model shape and pose parameters to canonicalize the medical data. Unlike SMPL, which is trained from upright 3D scans, MRI scans are acquired with subjects lying on a table, resulting in significant soft-tissue deformation. Consequently, HIT uses a learned volumetric deformation field that undoes these deformations. Since HIT is parameterized by SMPL, we can repose bodies or change the shape of subjects and the internal structures deform appropriately. We perform extensive experiments to validate HIT's ability to predict a plausible internal structure for novel subjects. The dataset and HIT model are available at https://hit.is.tue.mpg.de to foster future research in this direction.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Corresponding Author
Keywords Topography; Obesity
ISSN (print) / ISBN 1063-6919
Conference Title IEEE Conference on Computer Vision and Pattern Recognition
Conference Date 16-22 June 2024
Conference Location Seattle, WA
Quellenangaben Volume: , Issue: , Pages: 3480-3490 Article Number: , Supplement: ,
Publisher Ieee Computer Soc
Publishing Place 10662 Los Vaqueros Circle, Po Box 3014, Los Alamitos, Ca 90720-1264 Usa
Non-patent literature Publications
Institute(s) Institute of Diabetes Research and Metabolic Diseases (IDM)
Grants ANR JCJC SEMBA project
InternationalMax Planck Research School for Intelligent Systems