Assessing Surgical Skill in Orthopaedic Trauma Surgery Training: Behavioral Metrics for Digital Performance...

Surgical skill assessment in orthopaedic trauma surgery still relies on subjective expert ratings, which limits consistency and scalability. While digitalization offers a path toward objective and scalable assessment, the highly manual and haptic nature of surgery makes tool use difficult to capture digitally, keeping such approaches underdeveloped. This study introduces a digital assessment framework for orthopaedic trauma training that derives digital behavioral metrics (DBM) from tracked surgical tool motion. Specifically, it investigates (1) which DBM indicate technical proficiency and (2) to what extent these DBM are capable of predicting expert-rated surgical performance.

Twenty-eight participants performed 3 standardized fracture fixations on synthetic bone models of the radius, ulna, and fibula. Tool motion was captured and transformed into a digital twin from which metrics such as path length, smoothness, and task duration were derived. These metrics were statistically compared with the average Global Rating Scale (GRS) obtained from 4 experts who rated each surgical performance. (1) Correlation analysis identified skill-relevant metrics and (2) a predictive model was trained to estimate performance from DBM evaluating its accuracy against the individual expert ratings.

(1) Several DBM were found to be indicative of surgical performance. Measures based on tool path length and time per activity showed strong correlations with expert ratings, reaching coefficients of up to 0.6. Correlation strength varied across tools and procedures. (2) The predictive model achieved a mean absolute difference of 3.8 points from the average GRS score (scale range: 28-70), outperforming the mean interexpert difference of 4.6 points.

DBM were identified as valid indicators of surgical skill. The study further demonstrated their predictive value, showing closer alignment with experts' average GRS score than individual expert ratings. These findings highlight the feasibility of objective, expert-independent performance assessment in orthopaedic trauma surgery training.