Measurement and prediction of diagnostic performance during radiology training

Faculty evaluations of residents' diagnostic performance in radiology subspecialty rotations were examined in two studies in order to ascertain whether the relative capabilities of residents change during training and to predict residents' diagnostic performance using a three-dimensional form perception test. In the first study, numeric ratings by faculty members were averaged to provide interpretive/diagnostic scores for each of 16 residents in each of 5 consecutive half-years. Sixty-seven percent of the relative differences among residents' diagnostic proficiencies persisted during training. The magnitude of these unchanging differences between individuals in diagnostic performance strongly favors resident selection based upon diagnostic potential. An additional 22% of the relative differences correlated with an abrupt change in the rank order of residents in diagnostic performance at the beginning of the second year of residency. This rearrangement of ranks may have resulted from an abrupt change in the tasks and expectations assigned to residents. In the second study, correlations between scores on a recently described Form Test and monthly faculty ratings of diagnostic performance were computed. An average diagnostic performance score for each resident was generated for all rotations combined, each of eleven subspecialties, and all rotations completed within a given half-year. Scores on the Form Test correlated well with these combined diagnostic scores, further substantiating results reported previously. Form Test scores were highly correlated with subspecialty diagnostic performance scores in subspecialties using cross-sectional imaging methods, such as neuroradiology. Although Form Test scores were poorly correlated with half-year diagnostic performance during the first year, they were highly correlated with performance beginning in the second year.