Evaluation of projection images for visual quality control of automated left and right lung segmentations on T1-weighted MRI in large-scale clinical cohort studies.
Background/Objectives: To assess diagnostic accuracy of
two-dimensional (2D) projection methods for rapid visual quality control
of automated volumetric (3D) lung segmentations compared with
slice-based 3D review of segmentation results for application in
large-scale studies. Methods: Segmentation of right and left
lungs on T1-weighted MRI of 300 participants of the German National
Cohort (NAKO) study was performed using the nnU-NET framework. Three
variants of 2D projection images of segmentation masks were created:
maximum intensity projection (MIP) using pseudo-chromadepth encoding
with different color spectra for right and left lung (Colored_MIP) and
standard deviation projection of segmentation mask outlines, encoded in
black-and-white (Gray_outline) or using color-encoding
(Colored_outline). The worst of two ratings by two independent raters
conducting slice-based review for segmentation errors on underlying
imaging data and review for mislabeling errors served as the standard of
reference. All variants were evaluated by five raters each for
identification of segmentation errors and the majority rating was used
as index test. The time required for review was recorded and diagnostic
accuracies were calculated. Results: Sensitivities of
Colored_MIP, Colored_outline and Gray_outline were 88.2% [95%-CI 78.7%;
94.4%], 89.5% [80.3%; 95.3%] and 78.9% [68.1%; 87.5%]; specificities
were 98.7% [96.1%; 99.7%], 96.4% [93.1%; 98.5%] and 98.7% [96.1%;
99.7%]; and F1-scores were 0.918, 0.895 and 0.863, respectively. Mean
time per case and rater required for evaluation was 2.8 ± 0.9 s for
Colored_outline, 1.7 ± 0.1 s for Colored_MIP, and 2.0 ± 0.4 s for
Gray_outline. Conclusions: The 2D segmentation mask projection
images enabled the detection of segmentation errors of automated 3D
segmentations of left and right lungs based on MRI with high diagnostic
accuracy, especially when using color-encoding. The method enabled
evaluation within a matter of seconds per case. Segmentation mask
projection images may assist in visual quality control of automated
segmentations in large-scale studies.