Mind the Gap: Human and AI Uncertainties in Cardiac MRI Segmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Cardiology

Background:

• Cardiac MRI (CMRI) segmentation is crucial for diagnosis.
• AI models offer potential for automated segmentation but require reliability assessment.
• Understanding uncertainty in AI segmentation is key for clinical adoption.

Purpose of the Study:

• To quantitatively and qualitatively analyze human-annotated vs. AI-derived uncertainty in CMRI segmentation.
• To enhance the reliability of AI-based CMRI segmentation models.
• To foster improved human-AI collaboration in cardiac image analysis.

Main Methods:

• Utilized a dataset of 483 CMRI scans with clinician-provided segmentation masks and uncertainty scores.
• Employed a fuzzy-based algorithm to derive AI-based uncertainty metrics (class-wise, slice-wise, subject-wise).
• Compared AI-derived uncertainty with human-annotated scores and conducted qualitative clinician assessments.

Main Results:

• A strong inverse correlation was observed between AI-derived uncertainty and Dice score, indicating lower uncertainty predicts higher segmentation quality.
• Human-annotated uncertainty aligned with AI-derived uncertainty for specific structures (e.g., papillary muscle).
• Clinicians utilize prior knowledge for uncertainty scoring, a capability currently lacking in data-driven AI models.

Conclusions:

• AI-derived uncertainty can function as an effective quality control measure for CMRI segmentation.
• Current AI models lack the human ability to integrate structural and contextual information for uncertainty estimation.
• Further development is needed to bridge the gap in AI's contextual understanding for uncertainty assessment.