AI-driven qualitative skill assessment in laparoscopic training: a prospective observational study

Qualitative assessment of surgical skills is central to proficiency-based training but remains difficult to scale due to its reliance on expert evaluation. While simulator-derived metrics offer objective data, they largely fail to capture qualitative aspects of performance. Artificial intelligence (AI)-based video analysis may help bridge this gap by enabling standardized and reproducible assessment. In this prospective observational study conducted during the 41st Annual Davos Surgical Course (2024), 50 junior surgical residents performed laparoscopic cholecystectomy on porcine simulation models. Procedures were video recorded, anonymized, segmented, and independently evaluated by expert raters and an AI-based model using the Global Operative Assessment of Laparoscopic Skills (GOALS). The AI demonstrated excellent test-retest reliability for the overall GOALS score (ICC 0.91) and good reliability across most domains. Agreement between AI and expert ratings was also excellent for the total score (ICC 0.92), with good to excellent concordance across individual domains and minimal bias. These findings suggest that AI-based video analysis provides reliable and reproducible qualitative assessment of laparoscopic skills and may facilitate scalable integration of qualitative evaluation into surgical training and other video-based clinical assessment settings.