Validation of the American College of Radiology Bone Reporting and Data System™ (ACR Bone-RADS™) for classifying...

To validate the performance of the American College of Radiology (ACR) Bone-RADS™ in identifying malignant osteolytic bone tumors and to evaluate the inter-observer agreement between musculoskeletal radiologists performing ACR Bone-RADS™ assessments.

This single-center, retrospective study included patients who visited our orthopedic oncology department on January 2019-December 2020 for bone tumors. Three radiologists evaluated the initial radiographs and assessed whether the tumor was eligible for ACR Bone-RADS™ assessment considering its transverse location and radiodensity. For eligible tumors, the radiologists rated the descriptors of ACR Bone-RADS™ and calculated the Bone-RADS™ categories. Using multi-reader, multi-case analysis, the performance in identifying malignant bone tumors was assessed in terms of sensitivity and area under the receiver-operating-characteristic curve (AUC), while dichotomizing the Bone-RADS™ categories into potentially benign (categories 1-2) and potentially malignant (categories 3-4). Gwet's AC1 was calculated to evaluate inter-observer agreement on the categories. The proportion of malignancy in each Bone-RADS™ category was calculated.

In total, 278 patients (mean age ± standard deviation, 40 ± 22 years; 161 men; and 93 with malignant tumors) were eligible for assessment. The sensitivity and AUC for discriminating malignancy using ACR Bone-RADS™ were 95.0% and 0.915, respectively. There was moderate inter-observer agreement on the ACR Bone-RADS™ category, with a Gwet's AC1 of 0.594 (95% confidence interval, 0.536-0.652). The proportions of malignancy were 0.0-1.2% in Bone-RADS™ category 1; 4.0-9.0% in category 2; 18.2-30.3% in category 3; and 77.7-83.3% in category 4.

The ACR Bone-RADS™ showed high diagnostic performance in detecting malignant osteolytic bone tumors, with moderate inter-observer agreement.