Related Experiment Videos
Validation of expected objective performance indicators in surgical data analysis
Ye Li1, Sreeram Kamabattula1, Kiran Bhattacharyya1
1Advanced Product Development, Intuitive Surgical, Inc., Peachtree Corners, Georgia, United States of America.
Plos One
|August 7, 2026
Summary
A new probabilistic method for calculating Objective Performance Indicators (OPIs) in robotic surgery shows reduced bias and variability compared to conventional methods, improving skill assessment.
Area of Science:
- Robotics in Surgery
- Surgical Skill Assessment
- Computational Anatomy
Background:
- Objective Performance Indicators (OPIs) are crucial for quantifying surgical activity and assessing surgeon skill, particularly in robotic-assisted surgery.
- Conventional OPI calculation methods face challenges due to inherent variability in surgical procedures and step annotations.
- This variability can lead to ambiguous and unreliable OPI values, hindering accurate skill assessment.
Purpose of the Study:
- To validate a novel probabilistic framework for computing expected Objective Performance Indicators (OPIs).
- To compare the robustness and reliability of this probabilistic method against the conventional approach using a large clinical dataset.
- To assess the impact of annotator variability on OPI values derived from both methods.
Main Methods:
- Utilized a large clinical dataset comprising 5408 annotated steps from 1016 robotic surgery cases across seven procedure types.
- Compared a previously proposed probabilistic method for expected OPI calculation with the conventional OPI computation approach.
- Conducted analyses to evaluate biases, bias magnitudes, and variability of OPI values, specifically considering annotator noise.
Main Results:
- The probabilistic method yielded OPI values with less frequent bias (49.37%) compared to the conventional method (56.63%).
- Bias magnitudes were smaller when using the probabilistic approach.
- The probabilistic method demonstrated significantly less variability in OPI values in 40.56% of comparisons, versus 8.52% for the conventional method.
Conclusions:
- The probabilistic approach for calculating expected OPIs is more robust to annotator variability, exhibiting reduced bias and variance.
- This method offers improved reliability for computational analysis and conceptual interpretation of surgical data.
- The validated probabilistic framework shows significant promise for enhancing objective surgical skill assessment in robotic procedures.