Radiotherapy portal verification: an observer study

In many radiotherapy facilities radiotherapy portal verification is currently a subjective process based on the visual comparison of a treatment or portal image with a prescription or simulation image. The reliability of this process is unknown. We describe here a study in which 16 observers (oncologists, physicists and therapists) independently evaluated the geometric accuracy of 530 treatment fields on 45 patients. The treatment images were acquired by the BEAMVIEW on-line portal imaging system (Siemens Medical Laboratories, Concord, CA, USA). Illustrative examples of the large variation in observers' assessments of the same field are given. The kappa statistic is used to evaluate the degree of agreement between observers and between on-line (at the treatment unit) and off-line (in a quiet viewing room) assessments. The best interobserver agreement was between the four oncologists contributing to the study although this level of agreement was rated only as "fair". Comparison of on-line and off-line decisions made by therapists exhibited "poor" agreement. This study has provided statistical confirmation of the suspicions of many workers in the field of radiotherapy portal verification, viz that the subjective evaluation of field accuracy is unreliable. However, the degree of unreliability is surprisingly large. The inconsistencies between observers documented in this study need to be clearly acknowledged in the development of protocols for the clinical use of on-line portal imaging systems. Acceptable reliability in radiotherapy portal verification will only be achieved when subjective decision making is eliminated.