The effect of commercial coaching courses on performance on the MCAT

Two studies on the relationship between taking a commercial coaching course and performance on the Medical College Admission Test (MCAT) are reported. In both studies, one based on repeating examinees and another based on first-time examinees, differences in scores favoring coached examinees were found on the Science Knowledge and Science Problems subtests. The results on the Skills Analysis: Quantitative subtest were mixed, while neither study found differences between the two groups in scores on the Skills Analysis: Reading subtest. The analyses further revealed that the differences in scores on the Science Knowledge and Science Problems subtests were less pronounced for examinees with low scores on the Skills Analysis: Reading and Skills Analysis: Quantitative subtests, examinees with very low or very high undergraduate grade-point averages, and examinees from very selective undergraduate colleges. The substantive importance of the differences was also explored. The coaching effect was estimated to increase an average student's probability of acceptance by 5 percentage points. The results suggest that coaching for the MCAT has a limited effect that supports rather than undermines the validity of the test.