Investigating a self-scoring interview simulation for learning and assessment in the medical consultation

Area of Science:

• Medical Education
• Health Professions Education
• Simulation-Based Learning

Background:

• Simulated patients enhance medical consultation skills training for undergraduates.
• Adaptive simulations are emerging as a tool for automated learning and assessment in medical education.

Purpose of the Study:

• To assess the validity and reliability of adaptive simulations for medical consultation skills assessment.
• To explore the feasibility of using IT data analytics for automated assessment in medical training.

Main Methods:

• A generalizability (G) study was conducted using IT data analytics from medical students interacting with adaptive simulations.
• The G study focused on two variables: clinical decision points and communication skills.
• Analysis explored true score variance and reliability levels for automated scoring.

Main Results:

• The generalizability study for communication skills showed low true score variance.
• Assessment of clinical decision points yielded reliability levels comparable to rater-based scoring.
• Data analytics confirmed user knowledge of the Calgary-Cambridge model of consultation.

Conclusions:

• Adaptive simulations demonstrate potential as a valuable tool for both teaching and assessing medical consultation skills.
• Automated assessment of clinical decision-making within adaptive simulations is feasible and reliable.
• Further development is warranted to enhance the reliability of communication skills assessment in these simulations.