Related Experiment Videos
Artificial intelligence teaching assistants: a scalable solution for supporting struggling medical students
Alina Sami1, Mark Adkins1, Anne McLeod1,2
1MD Program, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Purpose:
Large language models (LLMs), such as OpenAI's ChatGPT, have demonstrated tutoring benefits in small-scale pilot studies within focused areas of medical education. This study evaluated the large-scale implementation of AI-teaching assistants (AI-TAs) within a compulsory medical school course.
Method:
A quasi-experimental observational study with mixed methods was conducted to assess the impact of AI-TAs in a compulsory first-year medical school course at the University of Toronto in 2024. The research team developed AI-TAs using OpenAI's ChatGPT-4o and introduced them as a supplementary resource at the course's midpoint. They analyzed exam performance among the students who used AI-TAs (n = 87) and students who did not (n = 206). Additionally, surveys (n = 18) and interviews (n = 10) explored student perceptions of AI-TAs' effectiveness, usability, and impact on learning.
Results:
Students who would later adopt AI-TAs had significantly lower pre-intervention exam scores than their peers [83.8% vs 88.1%; t(118.8) = -3.82, P < .001]. They also more often failed to meet the course's assessment standard pre-intervention (24.1% vs 6.4%). After AI-TA adoption, performance converged for both early users [86.1% vs 86.5%; t(95.63) = -0.35, P = .72] and late adopters [85.6% vs 86.5%; t(26.08) = -0.52, P = .61], with similar proportions falling below the course's standard (4.4%-6.4%). Thematic analysis and surveys identified 3 key advantages of AI-TAs: (1) reliable and accurate educational support, (2) efficient application across learning activities, and (3) improved psychological safety, promoting engagement in active learning.
Conclusions:
AI-TAs correlated with improved exam performance, fewer students in academic difficulty, and enhanced student engagement through a psychologically safe learning environment. These findings suggest that AI-TAs can serve as a scalable, cost-effective tool to support struggling students, while complementing traditional instruction.
