Reconceiving Safety Regulation for AI and ML Medical Software

Area of Science:

• Health Policy
• Medical Informatics
• Regulatory Science

Background:

• Current regulatory models for AI/ML clinical decision support tools are based on outdated twentieth-century frameworks.
• These existing models, such as treating software as a medical device or viewing AI as a sociotechnical system, have proven ineffective in addressing novel AI risks in healthcare.
• The 21st Century Cures Act of 2016, which favored a sociotechnical approach, has not been successfully implemented.

Purpose of the Study:

• To explore the challenges in regulating artificial intelligence (AI) and machine learning (ML) clinical decision support tools.
• To propose novel conceptualizations for AI/ML healthcare regulation beyond existing twentieth-century models.
• To ensure patient safety and the integrity of medical judgment in AI-assisted healthcare.

Main Methods:

• Analysis of existing regulatory models for medical software and AI in healthcare.
• Conceptualization of AI/ML tools through the lens of the corporate practice of medicine doctrine.
• Development of an "intelligent agent" conceptualization for AI/ML tools within the healthcare system.

Main Results:

• Twentieth-century regulatory models are inadequate for addressing the unique risks posed by AI and ML in clinical decision support.
• Current frameworks neglect the essential roles of healthcare professionals and oversight bodies in ensuring AI safety.
• New conceptualizations are needed, considering the potential for corporate actors to undermine physician judgment and AI's disruptive "colonization" of healthcare.

Conclusions:

• Effective regulation of AI/ML in healthcare necessitates moving beyond repurposed old frameworks.
• Protecting patient safety, healthcare culture, and values requires institutional and regulatory reforms.
• Innovative approaches are crucial to manage the complexities and potential harms of AI-enabled healthcare.