Comparing checklists and databases with physicians' ratings as measures of students' history and physical-examination...

To compare two methods of rating students' performances on history and physical examination: (1) by using checklists completed by standardized patients (SPs) and databases completed by students, and (2) by using ratings of students by three physicians for each SP-student encounter.

Four cases were chosen for the study, and 30 students were examined per case. The students were all in their fourth year at the Southern Illinois University School of Medicine in the spring of 1991. Two of the cases had both checklists and databases, and the remaining two had databases only. Each SP-student encounter was videotaped and was viewed independently by three physicians unfamiliar with the contents of the checklists and databases. The physicians' pooled ratings were then compared with the checklist and database scores. Uncorrected and corrected correlations were obtained, with the generalizability coefficient used as the index of reliability.

Interrater generalizability of physicians' ratings was very good, ranging from .65 to .93 for overall ratings. Generalizability of physicians' ratings pooled across the four cases was .85. Checklist scores tended to correlate higher with physicians' ratings than did database scores: across the cases, correlation coefficients between physicians' ratings and checklist scores and database scores were .65 and .39, respectively.

The checklist scores correlated strongly with the physicians' ratings of history and physical-examination skills, providing some evidence of validity for their use. The checklist scores correlated much better with the physicians' ratings than did the database scores. Possible explanations for this finding are discussed.