Related Experiment Video
Updated: Jun 29, 2026
Objectively Assessing Sports Concussion Utilizing Visual Evoked Potentials
Published on: April 27, 2021
It's not your device, it's how you use it - Sensitivity and specificity of PSQA devices in the SEAFARER head and neck
Lauren May1, José Antonio Baeza-Ortega1, Mohammad Hussein2
1School of Information and Physical Sciences, University of Newcastle, Newcastle, Australia.
Background:
There is large variability in patient specific quality assurance (PSQA) devices, techniques and protocols used clinically in radiotherapy. The SEAFARER methodology enables centres to evaluate their PSQA system's ability to detect intentionally introduced errors and compare their findings with community results.
Aim:
This study assessed the performance of PSQA systems in detecting simulated delivery errors for a head and neck radiotherapy case.
Methods:
A commercial dose-mimicking algorithm was employed to create plans of similar quality and robustness for common linac models based on an established plan. Using Python scripting, eleven copies of these plans were created and modified to simulate different delivery errors which were classified based on their dosimetric impact as 'should-pass' or 'should-fail'. Participating centres were asked to use their clinical PSQA methodologies to determine which of the modified plans passed their PSQA processes when compared to the unmodified plan's dose prediction.
Results:
Eighty-nine submissions were received from 44 centres. Using their clinical PSQA systems, 48% (21/44) of centres passed at least one 'should-fail' plan. Overall sensitivity and specificity across all devices were 79% and 82%, respectively. Multiple centres reported a review or change of their PSQA protocol. Five common PSQA devices (used for 5+ submissions) were included in a comparison. Sensitivities ranged 0-100%, with every common device having at least one perfect score of 100% sensitivity and 100% specificity.
Conclusion:
Sensitivity and specificity of PSQA systems varied greatly between centres. User choices rather than inherent characteristics determine the performance of common PSQA devices.
