Are there sex biases in standardized tests of radiation oncology knowledge?

Recent studies have identified biases directed against women in standardized tests. We tested for the existence of such biases in the American College of Radiology (ACR) In-Training Examination in Radiation Oncology and the American Board of Radiology (ABR) Written Radiation Oncology Board Examination.

Our request to the ABR to permit us to study performance on their examinations, as a function of sex, was refused. We obtained scores, through the cooperation of six academic radiation oncology departments, for residents-in-training taking the in-service examination and candidates taking the written board examination for the first time. Test results for 1984 to 1995 were blinded as to name, but not sex or institution of training. For the in-service examination, scores are reported as percentiles normalized to the year of training. The effect of multiple scores for the same resident was assessed using a repeated-measures analysis of variance. Residents were nested within each sex/institution combination and crossed with training year and calendar year. The effects of three factors (sex, institution, and year the examination was taken) on the results of the biology, physics, and clinical sections were evaluated with an analysis of variance. The interactions of sex with institution and year were included to determine the scope of the sex effect. For the board examination, scores are reported as percentiles, as well as an overall pass/ fail outcome. An analyses of variance was performed similar to that used for the in-service examination. In addition, Fisher's exact test and logistic regression were used to analyze overall outcome (pass/fail).

We obtained data for 79 residents (48 men and 31 women, 1.54:1) who took the in-service examinations 165 times. Sixty-two residents (41 men and 21 women, 1.95:1) had an initial sitting for the ABR written examination. On the in-service examination, for the biology, physics, and clinical subsections, calendar year, training year, and sex did not have a significant effect on examinees scores. Institution of training had a significant effect (P < .02) on the scores in biology and physics. The total in-service examination scores were not significantly influenced by calendar year, training year, or sex. Institution of training has a strong influence on overall score (P = .03) and the interaction of sex with training year is near significance level (P = .06). The power for our statistical tests ranged from 0.88 to 0.99. On the board examination, sex, institution of training, year the examination was taken, and interaction of sex with year or sex with institution of training did not have a significant effect on test scores. Pass rates were 90% for men versus 81% for women (P = .43).

Sex did not significantly influence the results of the in-service examination or the written board examination. Institution of training is the strongest influence on the results of the in-service examination.