AI-Generated Multiple Mini Interview (MMI) Stations for Medical School Admissions: Psychometric Evaluation

Area of Science:

• Medical Education
• Artificial Intelligence in Admissions
• Psychometrics

Background:

• Multiple mini interviews (MMIs) are standard for assessing nonacademic attributes in medical school admissions.
• Developing high-quality MMI stations is resource-intensive and relies on expert input.

Purpose of the Study:

• To evaluate the effectiveness of AI in generating MMI stations for medical school admissions.
• This study is the first empirical evaluation of AI-generated MMI stations in a real-world admissions context.

Main Methods:

• 56 MMI stations (17 AI-generated, 39 traditional) were assessed for 824 domestic applicants.
• Station quality was evaluated based on reliability (Cronbach's α) and discrimination capability (SD, range).

Main Results:

• AI-generated stations showed slightly higher reliability (α=0.82) than traditional stations (α=0.81), not statistically significant.
• Both methods had variable discrimination; AI stations had a higher proportion classified as optimal.
• AI-generated stations achieved psychometric performance comparable to traditionally developed stations.

Conclusions:

• AI is a valuable tool for generating MMI stations, offering scalability and potentially reducing faculty burden.
• AI can maintain or enhance the psychometric quality of MMI stations.
• Continuous quality assurance is vital for fairness and validity in admissions.