Using artificial intelligence to assess personal qualities in college admissions.

Area of Science:

• Educational Psychology
• Artificial Intelligence
• Higher Education Admissions

Background:

• Personal qualities like leadership and prosocial purpose are crucial for college success.
• Holistic assessment of these qualities in admissions is complex and resource-intensive.
• Artificial intelligence (AI) offers potential for efficient assessment but faces concerns regarding bias and transparency.

Purpose of the Study:

• To investigate an AI approach for assessing personal qualities in college admissions essays.
• To determine if AI can accurately reproduce human assessments across demographic subgroups.
• To evaluate the predictive validity of AI-generated scores for college graduation.

Main Methods:

• Human raters (research assistants, admissions officers) identified seven personal qualities in 3,131 applicant essays.
• Pretrained language models were fine-tuned with human ratings to create AI assessment tools.
• The AI's performance was validated on a national sample of 309,594 students.

Main Results:

• The fine-tuned AI models successfully replicated human coding of personal qualities across demographic subgroups.
• AI-generated scores demonstrated incremental validity in predicting 6-year college graduation rates.
• The AI approach offers a scalable and potentially equitable method for holistic review.

Conclusions:

• AI can be effectively utilized to assess personal qualities from applicant essays in college admissions.
• This AI methodology shows promise in enhancing the efficiency and validity of holistic review.
• Further research should address challenges and opportunities for AI in high-stakes educational assessments.