Audience Response System Facilitates Prediction of Scores on In-Training Examination

Audience response systems (ARS) are increasingly popular; however, their contribution to education is not completely clear. Our study found that scores from review quizzes delivered by an ARS correlate with in-training exam (ITE) scores and are viewed positively by residents. This information may be useful in identifying poor performers early so that targeted educational interventions can be made. The objective was to determine if scores on review quizzes delivered by an ARS correlate with ITE scores and to obtain participant feedback on use of the ARS for ITE preparation.

This was a prospective observational study of emergency medicine (EM) residents at six accredited EM residency programs. Subjects included residents who had taken previous ITEs. Subjects participated in bimonthly review sessions using an ARS. Twelve review quizzes were administered, each consisting of 10 multiple-choice questions. After the ITE, subjects completed an attitudinal survey consisting of six Likert-scale items and one "yes/no" item. We used a mixed linear model to analyze the data, accounting for prior 2012 ITE scores and nesting due to institution.

Among 192 participants, 135 (70.3%) completed the ITE in both 2012 and 2013; we analyzed their data for the first objective. Results from the mixed linear model indicate that the total mean score on the review quizzes was a significant [t(127) = 6.68; p < 0.001] predictor of the 2013 ITE after controlling for the 2012 ITE score. One hundred forty-six (76.0%) participants completed the attitudinal survey; 96% of respondents stated that they would like ARS to be used more often in resident education. Respondents felt the sessions aided in learning (mean 7.7/10), assisted in preparation for the ITE (mean 6.7/10), and helped identify content areas of weakness (mean 7.6/10).

Our results suggest that scores from review quizzes delivered by an audience response system correlate with in-training exam scores and is viewed positively by residents.