Related Experiment Video
Updated: Sep 17, 2025
Cone Beam Intraoperative Computed Tomography-based Image Guidance for Minimally Invasive Transforaminal Interbody Fusion
Published on: August 6, 2019
Objective Fluoroscopic Image-Based Assessment of Intraoperative Wire Navigation Skill Agrees with Subjective Expert
Dominik Mattioli1, Geb W Thomas1,2, Emily E Connor3
1Department of Industrial & Systems Engineering, University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA.
Background:
The current standard for assessing orthopedic technical skill demonstrated in operating room performance relies primarily on subjective evaluations administered by an expert mentor. This study demonstrates the utility of fluoroscopic image-based analysis as an objective mechanism for assessing technical proficiency for a common wire navigation procedure by comparing it for the first time to the gold standard in orthopedic skills assessment: subjective expert opinion.
Methods:
The final pin construct achieved during the closed reduction and percutaneous pinning of pediatric supracondylar humerus fractures was assessed for 23 operating room performances from fluoroscopic imaging to produce an objective ranking. Individual rank-orderings from six experts were independently aggregated into a consensus ranking for the same 23 performances. Inter-rater reliability of expert assessments was measured as Cronbach's α for individual rankings. Spearman correlation coefficients were used to evaluate relationships between individual expert rankings, a consensus ranking, and an algorithm ranking derived from objective scores.
Results:
The inter-rater reliability of the experts' individual rankings yielded an α of 0.78, exceeding the 0.70 threshold for acceptable reliability. There was strong agreement between the objective ranking and the expert consensus (R2 = 0.59), with the objective ranking agreement with consensus being superior to all but one individual expert.
Conclusion:
These findings suggest that objective fluoroscopic image-based analysis is an effective tool for assessing technical operating room performance and highlight its potential role as a complementary tool to expert assessment in orthopedic skills training.
Clinical Relevance:
While traditional assessments of intraoperative skill performance based on expert opinion remain important, they can be limited by cognitive biases and variability in feedback. The integration of objective metrics with expert consensus offers a more robust and scalable approach to skill assessment. This hybrid method has potential to complement subjective evaluations by facilitating more consistent and data-driven feedback, which can be particularly useful for training programs with limited mentor availability.
