Automatic Surgical Skill Assessment System Based on Concordance of Standardized Surgical Field Development Using...

Area of Science:

• Minimally Invasive Surgery
• Surgical Education
• Artificial Intelligence in Medicine

Background:

• Objective surgical skill assessment is crucial for training and reducing human burden.
• Standardization of surgical field development is a key aspect of skill assessment.
• Current manual video review is subjective and labor-intensive.

Purpose of the Study:

• To develop a deep learning model for recognizing standardized surgical fields in laparoscopic sigmoid colon resection.
• To evaluate the feasibility of automatic surgical skill assessment using this model.
• To assess the concordance of standardized surgical field development.

Main Methods:

• A retrospective study using intraoperative videos of laparoscopic colorectal surgery.
• Development of a deep learning model trained on videos from expert surgeons (Endoscopic Surgical Skill Qualification System [ESSQS] scores > 75).
• The model outputs an AI confidence score (AICS) representing similarity to standardized surgical fields.

Main Results:

• The AI confidence score (AICS) strongly correlated with the Endoscopic Surgical Skill Qualification System (ESSQS) score (Spearman correlation coefficient = 0.81).
• The model achieved high performance in screening low- and high-skill groups (Area Under the ROC Curve = 0.93 and 0.94, respectively).
• 650 videos were analyzed, with 60 used for model construction and 60 for validation.

Conclusions:

• The developed AI model demonstrates strong correlation with expert surgical skill scores, validating its use for automatic assessment.
• The model shows feasibility for an automated surgical skill screening system.
• Potential applications extend to other endoscopic procedures.