Need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.

Area of Science:

• Oncology
• Medical Imaging
• Artificial Intelligence

Background:

• Artificial intelligence (AI) shows promise for segmenting oncologic positron emission tomography (PET) images.
• Clinical translation requires performance assessment on relevant tasks, not just overlap metrics like the Dice score.

Approach:

• Investigated if Dice scores align with clinical task performance for AI segmentation of PET images.
• Retrospectively analyzed multi-center clinical trial data (ECOG-ACRIN 6668/RTOG 0235) for non-small cell lung cancer patients.
• Evaluated AI segmentation structures using both Dice scores and accuracy in quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG).

Key Points:

• Dice scores, measuring spatial overlap, may not correlate with clinical task performance.
• AI segmentation evaluation using Dice scores can yield inconsistent findings compared to task-based metrics.
• Task-based evaluation is essential for reliable AI performance assessment in quantitative PET.

Conclusions:

• Current evaluation metrics like Dice scores may not adequately represent AI segmentation performance for clinical tasks.
• Objective, task-based evaluation is necessary for AI methods in quantitative PET imaging.
• This study highlights the need for improved evaluation strategies for AI in oncologic PET analysis.