Reliability of faculty assessments of student case histories: a problem in chiropractic education

A competency based educational model necessitates reliable evaluation. This study examined faculty reliability in assessing students' case histories at one chiropractic college. Previous studies have found great variation in faculty assessment of clinical performance in medical residents. In this study, sixteen faculty members at the Canadian Memorial Chiropractic College evaluated three case histories by completing five rating scales on each one. The study found little faculty agreement in judging the case histories. Faculty evaluations ranged from extremely poor to extremely excellent for the same case history. In addition, the data indicate that the faculty's ratings were influenced by a halo effect. Evaluators made a general judgement, which influenced the ratings on the five scales. It is suggested that meetings of clinical instructors to standardize criteria of assessment may be beneficial.