Related Experiment Video
Updated: May 16, 2026
07:46
Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality
Published on: August 9, 2024
AI-Based Markerless Computer Vision Framework for Open Surgery Skill Assessment: A Prototype Assessment Framework
Alejandro Zulbaran-Rojas1, Mohammad Dehghan Rouzi1,2, Natasha Hansraj1,3
1Michael E. DeBakey Department of Surgery, Baylor College of Medicine, Houston, TX, USA.
Surgical Innovation
|May 14, 2026
Summary
This study shows an AI system can track hand motions from surgical videos, generating skill scores that align with expert evaluations. This technology offers a feasible way to assess surgical performance using kinematics.
Area of Science:
- Surgical Education
- Medical Technology
- Artificial Intelligence
Background:
- Artificial intelligence (AI) enables hand motion tracking from standard surgical video recordings.
- Translating these data into meaningful performance metrics remains challenging.
- Evaluating the validity of a markerless, AI-driven system for technical skill scoring in open surgery is crucial.
Purpose of the Study:
- To evaluate the preliminary validity of a markerless, AI-driven system for generating interpretable technical skill scores from an open-surgery task.
- To assess the correlation between AI-derived kinematic parameters and expert-rated surgical performance.
- To determine the feasibility of video-based kinematic scoring in surgical training.
Main Methods:
- Sixteen medical students and one instructor performed a one-handed knot-tying task.
- A deep learning algorithm tracked 21 hand joints from smartphone video recordings.
- Kinematic parameters were derived and grouped into economy of motion (EM), flow of motion (FM), and spatial organization (SO) domains, then correlated with expert assessments.
Main Results:
- AI metrics showed strong correlations with sensor-based measures (r = 0.79-0.88, P < 0.01).
- EM metrics correlated with product quality (PQ) and technical performance (TP) (r = 0.59-0.67, P < 0.01).
- Smoothness within the FM domain correlated with PQ and TP (r = 0.56-0.57, P < 0.01).
Conclusions:
- The AI framework translated hand kinematics into interpretable, cohort-normalized domain-level scores.
- These AI-derived scores aligned with expert assessment, supporting preliminary construct validity.
- Findings support the feasibility of video-based kinematic scoring for surgical training, warranting further studies on reliability and generalizability.