Automated and Artificial Intelligence (AI)-Derived Performance Assessment in Surgical Simulation: A Systematic Review

Area of Science:

• Medical Education
• Surgical Training
• Artificial Intelligence

Background:

• AI-assisted and automated performance assessment is growing in surgical education.
• The extent, effectiveness, and trustworthiness of AI in surgical skill assessment remain unclear.

Purpose of the Study:

• To systematically review the literature on AI-assisted and automated performance assessment in surgical education.
• To evaluate the efficacy, validity, and reliability of AI algorithms in assessing technical surgical skills.

Main Methods:

• Systematic review of studies from 2010-2025 in PubMed, Scopus, Embase, and IEEE Xplore.
• Included experimental and observational studies on AI for technical skill assessment in simulation and clinical practice.
• Extracted data on study characteristics, algorithms, performance measures, validity, reliability, and study quality.

Main Results:

• Twenty-nine studies met inclusion criteria, predominantly using supervised learning algorithms.
• Performance measures varied widely, with inconsistent documentation of validity and reliability.
• A shift towards simulation-based assessment and multimodal data sources was observed, but methodological heterogeneity persists.

Conclusions:

• AI-based assessment holds potential for surgical education, especially real-time adaptive feedback.
• Methodological inconsistencies, lack of transparency, and unproven validity/reliability limit generalizability.
• Standardized, validated methods are crucial for reproducible performance measures and ethical implementation in surgical simulation.