Related Experiment Video
Updated: Aug 2, 2025
Handheld Metal Detector Screening for Metallic Foreign Body Ingestion in Children
Published on: September 11, 2018
More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA
David Lyell1, Ying Wang1, Enrico Coiera1
1Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, NSW 2109, Australia.
Objective:
To examine the real-world safety problems involving machine learning (ML)-enabled medical devices.
Materials And Methods:
We analyzed 266 safety events involving approved ML medical devices reported to the US FDA's MAUDE program between 2015 and October 2021. Events were reviewed against an existing framework for safety problems with Health IT to identify whether a reported problem was due to the ML device (device problem) or its use, and key contributors to the problem. Consequences of events were also classified.
Results:
Events described hazards with potential to harm (66%), actual harm (16%), consequences for healthcare delivery (9%), near misses that would have led to harm if not for intervention (4%), no harm or consequences (3%), and complaints (2%). While most events involved device problems (93%), use problems (7%) were 4 times more likely to harm (relative risk 4.2; 95% CI 2.5-7). Problems with data input to ML devices were the top contributor to events (82%).
Discussion:
Much of what is known about ML safety comes from case studies and the theoretical limitations of ML. We contribute a systematic analysis of ML safety problems captured as part of the FDA's routine post-market surveillance. Most problems involved devices and concerned the acquisition of data for processing by algorithms. However, problems with the use of devices were more likely to harm.
Conclusions:
Safety problems with ML devices involve more than algorithms, highlighting the need for a whole-of-system approach to safe implementation with a special focus on how users interact with devices.
Insights
Real-world safety issues with machine learning (ML)-enabled medical devices often stem from data input problems. While device issues are common, user interaction problems are more likely to cause harm, necessitating a comprehensive safety approach.
Area of Science:
- Medical device safety
- Health informatics
- Machine learning in healthcare
Background:
- Machine learning (ML) is increasingly integrated into medical devices, raising concerns about real-world safety.
- Existing knowledge on ML safety often relies on theoretical limitations and case studies.
- Post-market surveillance data offers a valuable resource for understanding ML device safety issues.
Purpose of the Study:
- To systematically analyze real-world safety problems associated with machine learning (ML)-enabled medical devices.
- To identify the root causes and consequences of safety events involving ML medical devices.
- To inform strategies for the safe implementation of ML in healthcare.
Main Methods:
- Analysis of 266 safety events reported to the US Food and Drug Administration (FDA) MAUDE database (2015-2021).
- Categorization of events based on an existing Health IT safety framework to distinguish device vs. use problems.
- Identification of key contributors and classification of event consequences.
Main Results:
- Most safety events involved device-related problems (93%), primarily concerning data acquisition for ML algorithms (82%).
- However, use-related problems (7%) were significantly more likely to result in harm (RR 4.2).
- Event consequences ranged from potential harm (66%) to actual harm (16%) and healthcare delivery impacts (9%).
Conclusions:
- Safety issues with ML medical devices extend beyond algorithmic concerns, encompassing data input and user interaction.
- Problems related to the use of ML devices pose a higher risk of patient harm.
- A holistic, system-wide approach, emphasizing user-device interaction, is crucial for ensuring the safe implementation of ML medical devices.
Related Concept Videos
01:19Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
01:29Issues And Trends In Healthcare Delivery System
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
01:25Errors occurring during blood pressure monitoring
Several factors...
01:21Types of Reports II: Incident or Occurrence Report
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
01:30Bacterial Signaling
01:18Steps in Outbreak Investigation