Development and evaluation of an automated report-based chart checking tool in external beam radiotherapy

Computerized radiotherapy chart checking tools have revolutionized the initial physics chart review process as they offer verification of large amounts of plan parameters in seconds and allow physicists to concentrate on high-skill-level items that are challenging for automation. Both commercialized and in-house chart check solutions typically rely on access to the patient data within the live SQL database of the radiation oncology information system (ROS). This is a complex task potentially posing risks to both database integrity and overall system performance when accessed during clinic operation hours.

The aim of this study was to develop and test a chart checking tool based on AURA reports that utilize the reporting database to detect and analyze the errors occurring during treatment plan preparation in external beam radiotherapy.

An Automated AURA Report-based Chart Checking Tool (AARCCT) was developed using the python-based programming environment in the RayStation treatment planning system (TPS). Following TG-275 recommendations, the tool verifies specific physics check items in TPS plan data and Varian's ARIA ROS. The ARIA data was captured leveraging advanced Physics Summary (AURA) reports. The AARCCT was tested on > 600 patients receiving various modalities of treatment, including 3D-CRT, IMRT and electron treatments. In addition to applying the script to the plans immediately following plan development (i.e., before physics check), it was applied to ∼160 plans already reviewed by medical physicists. The detected errors were assessed with failure mode and effect analysis.

The AARCCT was able to analyze over fifty plan parameters in near to real-time. Before physics review, ∼48.6% of plans contained at least one error, largely low severity. The error rate was relatively constant throughout the year of testing. After physics checks were completed, AARCCT detected errors in ∼37.1% of physicists checked plans. Both before and after physics check, the most common errors were related to inaccuracies in patient setup imaging (24%), prescription (5.6%), written directive (6.3%), patient shifts (5.4%) and contours (2.6%). The error occurrence rate across the dosimetry team was found to be between ∼22% to 62% with no correlation to dosimetrist's experience. The relative error occurrence rate across the radiation oncologist (RO) team was found to be ∼40%-60%. The higher error rates were observed in ROs who were either recent hires or who had > 20 years of experience. Across the physics team, the error occurrence rate was ∼22%-47% with higher rates among those with less than three or more than twenty years of experience.

The AARCCT was found to be an essential tool to reduce error propagation following manual physics plan checks. The automated nature and omission of live ROS database access uniquely allows for smooth integration of the developed tool into the clinical workflow while minimizing the impact to ROS speed, functionality and security. The tool could also be used for evaluation of staff training and establishing uniformity of practice across the radiation therapy team members to improve quality and operational efficiency.