Critical performance behaviors of radiology residents: evaluation of two category systems

Previous research has suggested that certain behavioral aspects of job performance are critical for successful performance in a diagnostic radiology residency. We report two studies conducted to determine the long-term stability of critical performance behaviors for radiology residents and refine their definitions.

A trained psychology graduate student conducted critical incident interviews with 20 senior faculty in diagnostic radiology. From these interviews, the faculty generated 120 descriptions of exemplary or poor resident performance. These descriptors were then independently sorted by two radiologists into the previously defined categories of behaviors to evaluate consistency of the behaviors. As a second study, the 120 descriptors were sorted into an expanded behavioral definition system and the reproducibility, using the expanded system, was compared with the original results.

The interrater reliability for placing the current incidents in the originally described behavioral categories was good (Cohen's kappa 0.70). The overall distribution of incidents showed strong similarity to the original data. Sorting into the expanded categories improved the Cohen's kappa from 0.70 to 0.92, indicating that the expanded behavioral definitions improved reliability for categorizing a behavioral incident.

The critical behaviors necessary for successful performance in diagnostic radiology residency are stable over time. The expanded and refined system of definitions of these behaviors is more efficacious than the original system was.