Evaluating evidence-based medicine skills during a performance-based examination

To measure students' competencies in evidence-based medicine (EBM) skills [clinical decision making using evidence from published literature (content) and in transmitting clinical information to patients (communication)] within the context of a performance-based examination (PBE).

In 2002-03, under the direction of a Performance-Based Examination Oversight Committee, 16 EBM queries were developed for a pair of third-year PBEs. At the last station of the PBE, the standardized patient (SP) for that station asked a clinical EBM question relating to their "disease process." Students were asked to develop an appropriate clinical question, perform a Medline search for appropriate articles, critically appraise a complete selected article, reach a conclusion to their question, and transmit the information to the SP. Each student's clinical question, search terms, selected articles, and rationale were evaluated by faculty question-writers, clinical librarians, and the EBM course director using a five-point Likert scale, with 1 being inadequate performance and 5 being superior performance. The SP evaluated the communication skills using a checklist.

Students' performances were very good, with means of 3.7 to 4.0 in each area. Agreement between the course director and station developers was good. Seventy-five percent of the students performed adequate Medline searches. Students averaged over 93% on the performance of four communication skills.

The evaluation of EBM skills can be carried out during a performance-based examination. Results can assist in developing students' skills and directing curricular efforts.