Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal

To examine the evidence that the teaching of critical appraisal (evidence-based medicine) skills to undergraduate medical students or residents will result in significant gains in knowledge and increased use of the literature in clinical decision-making.

Articles published from 1966 to 1995, retrieved through a MEDLINE search supplemented by manual searches; review of bibliographies maintained by individuals involved in teaching critical appraisal skills; and a previous methodological review.

Articles were selected if the study involved some form of control group, although strict randomization was not required, and a measure of performance followed the intervention. Articles were excluded if they simply reported the process of teaching critical appraisal skills or used some form of "happiness index."

There were 10 studies of the impact of teaching critical appraisal skills, 6 involving medical students and 4 involving residents. Results from 3 of the studies were nearly uninterpretable and thus were excluded; the remaining 7 were methodologically acceptable. Analysis showed that interventions implemented in undergraduate programs resulted in significant gains in knowledge, as assessed by a written test (mean gain 17.0%; standard deviation [SD] 4.0%). Conversely, studies at the residency level consistently showed a small change in knowledge (mean gain 1.3%; SD 1.7%). Two studies that examined residents' use of the literature were unable to demonstrate any positive changes.

Studies of the effect of teaching critical appraisal skills on gains in knowledge at the undergraduate level showed consistent improvement. By contrast, changes in knowledge at the residency level were small. Several suggestions from the educational literature are offered to increase effectiveness of critical appraisal interventions.