A Statistical Framework to Detect and Quantify Operator-Learning Curves in Medical Device Safety Evaluation.

Area of Science:

• Medical device safety
• Health informatics
• Statistical modeling

Background:

• Post-market medical devices can present safety issues, leading to patient harm and increased healthcare costs.
• Learning effects (LE) in medical device usage are increasingly recognized as a significant factor influencing safety outcomes.
• Accurate differentiation between learning effects and inherent device issues is crucial for implementing targeted safety interventions.

Purpose of the Study:

• To develop and validate a statistical framework for detecting learning effects (LE) in medical device usage.
• To quantify the learning curve (LC) associated with the use of medical devices.
• To assess the framework's performance in distinguishing LE from device-specific safety signals.

Main Methods:

• Generation of synthetic datasets reflecting clinical data distributions and correlations.
• Application of generalized additive models to develop predictive models.
• Utilizing the Levenberg-Marquardt algorithm for estimating learning curve (LC) parameters.
• Evaluation of the framework's sensitivity, specificity, and likelihood ratio (LR) for LE detection.

Main Results:

• The framework demonstrated high sensitivity (90%) and specificity (88%) in detecting LE across simulated datasets.
• Accurate estimation of the learning curve (LC) was achieved in 81% of datasets where LE was present.
• The analytic framework proved robust in disentangling LE from device safety signals.

Conclusions:

• The developed framework effectively models and characterizes operator learning effects in medical device safety evaluations.
• This approach enables better attribution of safety signals, leading to improved patient safety recommendations.
• Further validation using real-world clinical datasets is recommended to confirm the framework's utility.