Validity and equity of the AAMC PREview situational judgment test in U.S. medical school admissions

Situational judgment tests (SJTs) are increasingly considered in medical school admissions to assess non-academic competencies within holistic review. Questions remain regarding accessibility, variation in performance across applicant groups, and their relationship to admissions outcomes. This study examines the association between AAMC PREview SJT scores and admissions outcomes in a real-world applicant pool, where scores were not used in decision-making and were reviewed retrospectively.

We conducted a retrospective analysis of 5,993 applicants to the Icahn School of Medicine at Mount Sinai during the 2022-2023 application cycle. Of these, 4,046 (67.5%) completed the PREview SJT. Scores were not available to the admissions committee at any stage and did not influence interview or acceptance decisions. Applicants were categorized by socioeconomic status (SES) and first-generation college status. Logistic regression models evaluated associations between PREview scores, interview invitations, and post-interview committee evaluations, adjusting for demographic variables. Outcomes were compared between test takers and non-takers.

Applicants from lower SES and first-generation backgrounds were more frequently represented among lower PREview score categories. PREview scores were not associated with post-interview committee evaluations. However, lower PREview scores were associated with a lower likelihood of interview invitation. Among interviewed applicants, neither PREview score nor demographic variables were associated with committee outcomes.

In a setting where PREview scores were not used in admissions decisions, lower scores were associated with interview invitation status but not with post-interview committee evaluations. These findings suggest limited alignment between PREview scores and the outcomes of holistic admissions review and underscore the importance of careful evaluation before incorporating SJTs into admissions processes.