The development and evaluation of a clinical test of surgical resident proficiency

The two purposes of this study were to develop a clinical test of surgical resident proficiency and to compare this clinical test with currently applied methods of resident evaluation. Appendicitis is a common surgical disease, and its accurate diagnosis depends largely on clinical acumen. Ten third-year surgical residents prospectively evaluated 107 patients admitted because of suspected appendicitis. After taking a history, performing a physical examination, and reviewing laboratory data, these residents were asked to state, as a percentage, the likelihood that each patient had appendicitis. Sixty-three patients had appendicitis documented by pathologic inspection after appendectomy. Forty-four patients did not have appendicitis, as was determined by operation (17) or by in-house observation and resolution of abdominal pain (27). These outcomes were used to calculate a diagnostic ability score (DAS) for each resident. Residents were evaluated by standard methods including in-service examinations and monthly evaluations by the attending staff. Residents were also evaluated by nonstandard, but potentially useful, neuropsychologic and psychologic tests, including the trail making test, the Ravens progressive matrices test, the paced auditory serial addition test (PASAT), the grooved pegboard test of manual dexterity, and the profile of mood states (POMS) psychologic questionnaire. Cumulative scores were calculated and compared by multiple regression with coefficient variance analysis. The correlation (R2) of DAS with standard evaluation techniques was as follows: In-service (0.055), faculty (0.508), trails (-0.293), Ravens (0.028), PASAT (0.251), dexterity (0.432), POMS (0.381). We found that (1) the DAS is a discriminating clinical test; (2) the DAS correlates with subjective faculty evaluation; and (3) the DAS does not correlate with in-service examination scores. We conclude that faculty evaluation remains the best currently applied test of surgical resident clinical proficiency as measured by the DAS.