Artificial intelligence for assessment in competency-based medical education: current practices and future directions

Area of Science:

• Medical Education Technology
• Artificial Intelligence in Healthcare
• Assessment Science

Background:

• Competency-Based Medical Education (CBME) faces challenges in consistent assessment implementation.
• Artificial Intelligence (AI) offers potential for improved assessment efficiency, objectivity, and feedback in CBME.
• Current understanding of AI practices and evaluation methods in CBME is limited.

Purpose of the Study:

• To map existing Artificial Intelligence (AI) applications in Competency-Based Medical Education (CBME) assessments.
• To guide future research and development of AI in medical education.
• To understand the current landscape of AI-driven assessment tools in medical training.

Main Methods:

• A comprehensive literature search was conducted across multiple databases (MEDLINE, EMBASE, PsycINFO, Scopus).
• Studies focused on AI deployment for assessment generation, analysis, or interpretation within CBME.
• PRISMA-ScR guidelines were followed for reporting, and findings were synthesized using Levac et al.'s approach.

Main Results:

• 32 studies met inclusion criteria from 1002 initial results.
• AI applications span surgical skills, clinical notes, communication, feedback generation, performance prediction, and narrative analysis.
• Diverse AI tools are being utilized across undergraduate, graduate, and continuing professional education.

Conclusions:

• AI integration in CBME offers advantages like timely evaluations but faces challenges such as lack of granularity.
• Thoughtful AI integration can complement traditional assessments and enhance learner outcomes.
• Successful AI implementation requires robust infrastructure, ethical oversight, and collaborative policy development.