An interim report on the MCAT Essay Pilot Project.

Area of Science:

• Medical Education
• Psychometrics
• Standardized Testing

Background:

• The Medical College Admission Test (MCAT) essay is a critical component of medical school admissions.
• Understanding factors influencing essay performance is essential for fair and valid assessment.

Purpose of the Study:

• To analyze MCAT essay performance across demographic groups.
• To examine the relationship between essay scores and academic/demographic characteristics.
• To evaluate the reliability and quality of different essay administration formats.

Main Methods:

• Pilot testing of the MCAT essay question across four administrations.
• Analysis of performance data stratified by gender, hometown size, race/ethnicity, and dominant language.
• Assessment of essay score correlations with academic and demographic variables.
• Comparison of reliability and writing quality between one 45-minute essay and two 30-minute essays.

Main Results:

• No significant performance differences were observed based on gender or hometown size.
• Racial/ethnic group differences in scores were largely attributable to reading level variations.
• Significant differences in essay performance were noted for dominant language groups, independent of reading level.
• Two 30-minute essays demonstrated higher reliability than one 45-minute essay, but with reduced writing quality.
• No correlation was found between essay scores and academic/demographic characteristics.

Conclusions:

• Dominant language is a key factor influencing MCAT essay performance.
• Future research should focus on test-retest reliability for 45-minute essays and performance of non-native English speakers.
• The impact of the essay on the overall medical school selection process requires further assessment.