Related Experiment Video
Updated: May 15, 2025
The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living
Published on: July 28, 2022
First validated automated scoring system using the diagnostic arthroscopy skill score (DASS 2.0) for assessing
Hermann Anetzberger1, Andreas Kugler2, Michael Mohr3
1Orthopädische Gemeinschaftspraxis am OEZ, Munich, Germany.
Purpose:
Proficiency-based simulator training is a promising approach for learning the complex motor skills required for arthroscopy. However, its implementation requires an objective assessment tool to assess residents' arthroscopic skills. To address this need, an automated diagnostic arthroscopy skill score (DASS) was developed and validated as a replacement for manual scoring.
Methods:
An automated measurement system of arthroscopic skills was developed, replacing the manual assessment parameters of the DASS with objective measurement criteria. Data from arthroscopies performed by 20 experts were used to establish threshold values for scoring. To validate the new method, 125 videos of residents were recorded and evaluated by five raters. The results of the automated evaluation were then compared with those of established manual assessment. To assess the reliability of the manual evaluation, the intraclass correlation coefficient (ICC) and minimum detectable change (MDC) were calculated. Methodological agreement was evaluated using linear least squares regression and the Bland-Altman method.
Results:
A good to excellent level of reliability was found among the five raters (ICC for DASSpart1 = 0.89, 95% confidence interval [CI] = 0.84-0.92). The calculated MDC was 5.0 points. High methodological agreement was found between the manual and automated evaluations of the DASS. The 95% CI for the slope of the regression line included 1, and the 95% CI for the intercept included 0. According to the Bland-Altman method, the mean difference between manual and automated evaluations was 4.1 ± 5.4 points, and the scattering of the measurement differences was uniformly distributed, regardless of the total score.
Conclusions:
Automated DASS measurement is a valid and reliable tool for assessing arthroscopic skills. Its advantages, such as rater independence, precise and objective measurements, and immediate evaluation, make it a powerful tool for evaluating arthroscopic performance during simulation training.
Level Of Evidence:
Level II.
