Need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.

Area of Science:

• Medical imaging
• Artificial intelligence
• Oncology

Background:

• Artificial intelligence (AI) shows promise for segmenting oncologic positron emission tomography (PET) images.
• Clinical translation requires performance assessment on relevant tasks.
• Current evaluation metrics, like Dice scores, may not correlate with task performance.

Purpose of the Study:

• To investigate if Dice scores align with clinical task performance for AI-based PET image segmentation.
• To compare Dice score evaluation with task-based quantification of metabolic tumor volume (MTV) and total lesion glycolysis (TLG).

Main Methods:

• Retrospective analysis of multi-center clinical trial data (ECOG-ACRIN 6668/RTOG 0235).
• Evaluation of AI segmentation methods using both Dice scores and accuracy in quantifying MTV/TLG.
• Comparison of results from Dice score evaluation versus task-based evaluation.

Main Results:

• Dice scores can lead to interpretations inconsistent with task-based performance.
• AI segmentation evaluation using Dice scores may not accurately reflect clinical utility for MTV/TLG quantification.
• Discrepancies highlight limitations of overlap metrics for clinical tasks.

Conclusions:

• Task-based evaluation is essential for AI-based segmentation methods in quantitative PET.
• Relying solely on Dice scores may misrepresent the clinical relevance of AI segmentation tools.
• Objective, task-specific metrics are needed for reliable clinical translation of AI in oncology PET.