Learning by doing: teaching critical appraisal of randomized trials by performing an in-class randomized trial

Our medical students often encounter difficulty because many have trouble appreciating the relative merits and limitations of clinical research design. In order to remedy these deficits, we developed a novel in-class simulated randomized trial in which students were blinded to their treatment assignment. Specifically, our goals for the session were to demonstrate the importance of randomization of subject populations, to define and use basic terminology such as relative risk reduction along with absolute risk reduction, and to demonstrate the strengths and pitfalls of analysis of subsets of subjects.

We proposed the study of a hypothetical new drug, "Flunkimmune," which was claimed to prevent students from failing tests. Students received a random assignment to either the "Flunkimmune" or the placebo group. Students then received an opaque plastic egg appropriate for their randomization. We had prepared the eggs a-priori to each contain a piece of candy, and some to contain a blue dot. The flavor of the candy signified whether a particular student was in the treatment or control group. After all students had received their eggs, we "unblinded" the study by having the students open their eggs and learn to which treatment they had received. We then used an audience response system to construct a demographics table commonly known as Table 1 for the randomized trial-based characteristics of the students' choosing (e.g., "drinks Diet cola," "over six feet tall," "birth weight under seven pounds"). We asked the class to critique their own Table 1 and to comment on the adequacy of the randomization. Students were then informed that the presence of the blue dot signified a "failing" grade on a hypothetical exam to test the efficacy of "Flunkimmune." The eggs had been loaded so that the treatment group had a relative risk reduction of failure rates of 50% compared with the control group and an absolute risk reduction of 20%. This result prompted a debate among students about the relative merits of absolute or relative risk reduction in reporting the results of the study. Students also performed subgroup analyses by determining the failure rates for treatment and control students seated in particular aisles of the lecture hall. This enhanced discussion about the processes of bias and confounding inherent in subgroup analyses.

This experiential teaching method allowed students to participate directly in a simulated randomized controlled trial, thereby facilitating understanding of this type of study. Students reported that the experience was enjoyable and stimulated active engagement with critical appraisal of randomized trials. Students' subsequent homework assignments, in which they were asked to critically appraise a randomized trial, demonstrated a depth of understanding not previously encountered at this level. We plan to develop and rigorously evaluate additional experiential activities that expose students to the role of the research investigator.