American Board of Radiology certifying examination: oral versus computer-based format

This pilot study of a computer-based examination for primary certification by the American Board of Radiology was designed to acquire comparative data on candidates that were measures of individual performance on the oral examination compared with the computer-based examination.

The pilot computer-based pediatric radiology examination was designed by experienced oral board examiners and the pediatric subspecialty trustees. Images were chosen from the examination repository of the American Board of Radiology. The 20-minute examination was designed to include 8-10 cases with 26-31 scorable units covering all aspects of pediatric radiology.

Among the 1,317 candidates taking the oral board examination, 1,048 candidates (79.6%) participated in the voluntary pilot examination. The scores of the two examinations were subjected to statistical analysis. The sensitivity and specificity of the pilot examination were 94.5% and 45.7%. The overall accuracy was 92.8%. Seventy-five candidates (7.2%) who participated in this study received different verdicts on the pilot examination and the pediatric radiology category of the oral examination. Fifty-six of these candidates (5.3%) failed the pilot examination but passed in the oral pediatric radiology category; 19 of the candidates (1.8%) passed the pilot examination but failed the oral pediatric radiology test. Pilot examination scores were higher for candidates who passed the oral pediatric radiology category (median score, 80; interquartile range, 74.1-85.2) than for candidates who failed (median score, 65.4; interquartile range, 58.6-71.0) (p < 0.0001).

The pediatric pilot examination was useful for differentiating passing candidates from failing candidates when the score in the pediatric radiology category of the oral examination was used as the reference standard. The overall accuracy was 92.8%.