Related Experiment Video
Updated: Sep 3, 2026
A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis
Published on: February 6, 2020
An interpretable whole-stream video based deep learning framework for surgical skill assessment of basic procedural
Ke Chen1,2, Ying Weng1,2, Yiming Zhang1,2
1School of Computer Science, University of Nottingham Ningbo China, Ningbo, China.
Background:
Conventional surgical skill assessment, conducted by expert surgeons in operating rooms, is often time-consuming and subjective. An automated and objective system capable of differentiating surgeon expertise levels (novice, intermediate, and expert) and providing feedback would significantly enhance surgical training. However, most existing automated frameworks prioritize deep learning performance over interpretability.
Materials And Methods:
We propose I3D-SAP, an interpretable video-based surgical skill assessment framework of basic procedural elements using a 3D convolutional neural network. The model processes surgical videos and classifies surgeons into three expertise levels. Experiments are conducted on the public JIGSAWS dataset, evaluating performance across suturing, needle passing, and knot tying tasks.
Results:
Our method has achieved high accuracy: 100% (suturing), 96.3% (needle passing), and 97.2% (knot tying), comparable to the state-of-the-art models. Our framework's interpretability is demonstrated through Class Activation Map (CAM) visualization and snippet-wise analysis, offering dual perspectives to explain classification decisions. However, evaluation under a more rigorous Leave-One-User-Out (LOUO) protocol revealed a significant performance drop, and testing on an external dataset (ROSMA) resulted in near-chance-level accuracy, indicating that the current model's ability to generalize to entirely unseen surgeons is limited.
Conclusion:
I3D-SAP provides both high accuracy and interpretability, addressing a critical gap in automated surgical assessment of basic procedural elements. By visualizing the model decisions, it enhances transparency and trust, paving the way for practical adoption in surgical training. Moreover, the model's current generalizability challenges should be addressed through future work involving larger, more diverse datasets and techniques for surgeon-invariant feature learning before practical clinical deployment can be considered.
