Reliability and accuracy assessment of Radiation Therapy Oncology Group-endorsed guidelines for brachial plexus contouring

The goal of this work was to validate the Radiation Therapy Oncology Group (RTOG)-endorsed guidelines for brachial plexus (BP) contouring by determining the intra- and interobserver agreement. Accuracy of the delineation process was determined using anatomically validated imaging datasets as a gold standard.

Five observers delineated the right BP on three cadaver computed tomography (CT) datasets. To assess intraobserver variation, every observer repeated each delineation three times with a time interval of 2 weeks. The BP contours were divided into four regions for detailed analysis. Inter- and intraobserver variation was verified using the Computerized Environment for Radiation Research (CERR) software. Accuracy was measured using anatomically validated fused CT-magnetic resonance imaging (MRI) datasets by measuring the BP inclusion of the delineations.

The overall kappa (κ) values were rather low (mean interobserver overall κ: 0.29, mean intraobserver overall κ: 0.45), indicating poor inter- and intraobserver reliability. In general, the κ coefficient decreased gradually from the medial to lateral BP regions. The total agreement volume (TAV) was much smaller than the union volume (UV) for all delineations, resulting in a low Jaccard index (JI; interobserver agreement 0-0.124; intraobserver agreement 0.004-0.636). The overall accuracy was poor, with an average total BP inclusion of 38%. Inclusions were insufficient for the most lateral regions (region 3: 21.5%; region 4: 12.6%).

The inter- and intraobserver reliability of the RTOG-endorsed BP contouring guidelines was poor. BP inclusion worsened from the medial to lateral regions. Accuracy assessment of the contours showed an average BP inclusion of 38%. For the first time, this was assessed using the original anatomically validated BP volume. The RTOG-endorsed BP guidelines have insufficient accuracy and reliability, especially for the lateral head-and-neck regions.