Automated planning target volume generation: an evaluation pitting a computer-based tool against human experts

Area of Science:

• Radiation Oncology
• Medical Physics
• Software Engineering in Healthcare

Background:

• Three-dimensional treatment planning software is increasingly used in radiotherapy.
• The clinical implementation of these tools often lacks rigorous pre-use evaluation.
• Automated tools for generating planning target volumes (PTVs) require validated methodologies.

Purpose of the Study:

• To demonstrate a rigorous evaluation methodology for automated radiotherapy software.
• To assess the performance of an automated software tool for generating ICRU-50 PTVs using blinded peer review.
• To compare the software's PTV generation capabilities against human expert performance.

Main Methods:

• Seven physicians from three institutions participated in the evaluation.
• Four physicians created partial PTVs on nine test cases (nasopharynx and lung primaries).
• An automated software tool generated PTVs using the same data, and three blinded evaluators assessed all PTVs for acceptability.

Main Results:

• The automated software tool demonstrated success comparable to human experts in PTV generation.
• Common failures included insufficient margins and encroachment on critical structures for both computer and human experts.
• Qualitative analysis revealed similar types and distributions of errors between the software and human experts.

Conclusions:

• Rigorous evaluation of computer-based radiotherapy tools is essential before clinical adoption.
• Comparison against current clinical practice, like expert-drawn PTVs, is crucial for identifying areas of improvement.
• Blinded peer review provides a robust method for evaluating automated radiotherapy planning software.