Evaluation of Two Commercial Artificial Intelligence Segmentation Systems for Radiation Therapy

Area of Science:

• Medical Imaging
• Radiation Oncology
• Artificial Intelligence

Background:

• Artificial intelligence (AI) is increasingly used in medical imaging for automated segmentation.
• Accurate organ delineation is crucial for effective radiation therapy planning.
• Evaluating AI performance against expert contours is essential for clinical adoption.

Purpose of the Study:

• To assess the clinical acceptability of AI auto-segmentation systems.
• To compare AI performance with physician expert delineated contours.
• To evaluate contour accuracy and volume differences between AI systems and experts.

Main Methods:

• Thirty-three head and neck cancer patients' CT scans were analyzed.
• Two AI systems (MIM Protégé AI, TheraPanacea Annotate) generated auto-segmented contours.
• Physician experts performed a Turing test and rated contour acceptability on a 3-point scale.

Main Results:

• Experienced experts could differentiate AI from manual contours; less experienced experts struggled.
• TheraPanacea outperformed MIM in Dice similarity scores for 60% of structures.
• Despite higher Dice scores, experts favored MIM, likely due to workflow integration and trust.

Conclusions:

• AI auto-segmentation enhances radiation therapy planning efficiency.
• This study highlights AI's role in streamlining organ delineation.
• Differences in vendor performance exist, but AI-driven contouring offers significant clinical benefits.