Related Experiment Video
Updated: Jan 4, 2026
Development of a Virtual Reality Assessment of Everyday Living Skills
Published on: April 23, 2014
A Study of the Validity of the New MCAT Exam
Claudio Violato1, Jacqueline L Gauer, Efrem M Violato
1C. Violato is professor and assistant dean for assessment and evaluation and affiliated with the Medical Education Outcomes Center, University of Minnesota Medical School-Twin Cities, Minneapolis, Minnesota. J.L. Gauer is manager, evaluation and analytics, and affiliated with the Medical Education Outcomes Center, University of Minnesota Medical School-Twin Cities, Minneapolis, Minnesota. E.M. Violato is a PhD candidate, University of Alberta, Edmonton, Alberta, Canada, and affiliated with the Medical Education Outcomes Center, University of Minnesota Medical School-Twin Cities, Minneapolis, Minnesota. D. Patel is associate dean for admissions and affiliated with the Medical Education Outcomes Center, University of Minnesota Medical School-Twin Cities, Minneapolis, Minnesota.
Purpose:
To conduct a study of the validity of the new Medical College Admission Test (MCAT).
Method:
Deidentified data for first- and second-year medical students (185 women, 54.3%; 156 men, 45.7%) who matriculated in 2016 and 2017 to the University of Minnesota Medical School-Twin Cities were included. Of those students, 220 (64.5%) had taken the new MCAT exam and 182 (53.4%) had taken the old MCAT exam (61 [17.9%] had taken both). The authors calculated descriptive statistics and Pearson product moment correlations (r) between new and old MCAT section scores. They conducteda regression analysis of MCAT section scores with Step 1 scores and with preclerkship course performance. They also conducted an exploratory factor analysis (principal component analysis with varimax rotation) of MCAT scores, undergraduate grade point average, Step 1 scores, and course performance.
Results:
The new MCAT exam section mean score percentiles ranged from 72 to 78 (mean composite score percentile of 80). The old MCAT exam section mean score percentiles ranged from 84 to 88 (mean composite score percentile of 83). The pattern of correlations among and between new and old MCAT exam section scores (range of r: 0.03-0.67; P < .01) provided evidence of both divergent and convergent validities. Backward multiple regression of new MCAT exam section scores and Step 1 scores resulted in a multiple R of .440; the same analysis with Human Behavior course performance as the dependent variable provided a similar solution with the expected sections of the new MCAT exam (multiple R = .502). The factor analysis resulted in 4 cohesive, theoretically meaningful factors: biomedical knowledge, basic science concepts, cognitive reasoning, and general achievement.
Conclusions:
This study provided empirical evidence of multiple types of validity for the new MCAT exam.
Related Concept Videos
01:29Reliability and Validity
01:23McNemar's Test
01:13Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:29Theory of Attribution II: Kelley's Covariation Theory
01:22Self-Report Tests of Personality
01:29Measures of Intelligence
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...