Evaluation of multiple-vendor AI autocontouring solutions

Area of Science:

• Medical Physics
• Radiotherapy
• Artificial Intelligence

Background:

• Artificial intelligence (AI)-based autocontouring tools promise improved accuracy and efficiency in radiotherapy planning.
• Evaluating commercially available AI solutions is crucial before clinical integration.

Purpose of the Study:

• To assess the accuracy and clinical utility of three AI-based autocontouring solutions.
• To compare AI-generated contours against physician-approved manual contours.

Main Methods:

• Three AI solutions (MIM-ProtégéAI+, Radformation-AutoContour, Siemens-DirectORGANS) were tested on 47 patients with 16 contoured organs.
• Physician scoring (Likert scale) and quantitative metrics (Dice Similarity Coefficient, Hausdorff Distance, Mean Distance to Agreement) were used for evaluation.
• AI contours were compared to physician-approved manual contours as the ground truth.

Main Results:

• Physician scores indicated AI contours were generally acceptable, with average scores close to manual contours (AP: 2.02, MIM: 2.07, RAD: 1.96, SIE: 1.99).
• 85.4% of AI contours achieved an average Dice Similarity Coefficient (DSC) ≥ 0.7.
• Quantitative metrics showed variability, with average Hausdorff Distance ranging from 2.9 to 43.3 mm and Mean Distance to Agreement from 0.6 to 26.1 mm.

Conclusions:

• All evaluated AI autocontouring solutions demonstrated performance comparable to manual contouring.
• Unusual anatomy presented challenges for AI, occasionally requiring manual adjustments and physician review.
• The comparable performance suggests centers can select AI solutions based on workflow integration and resource availability, with continued physician oversight recommended.