Does AI in Healthcare Need an Editor-in-Chief? A Leading Example of True Human Oversight

Area of Science:

• Healthcare AI
• Medical Ethics
• Regulatory Compliance

Background:

• High-risk Artificial Intelligence (AI) integration in healthcare offers efficiency but poses safety and accountability challenges.
• The European Union's AI Act mandates human oversight for healthcare AI, creating regulatory requirements.
• Current regulations lack clarity on ongoing responsibility for AI oversight, risking fragmented accountability and potential harm.

Purpose of the Study:

• To analyze the ambiguities in implementing human oversight for high-risk healthcare AI under the EU AI Act.
• To propose a systemic, institution-wide AI governance model for effective oversight.
• To advocate for a shift from "Human-in-the-Loop" to multidisciplinary AI Governance Committees.

Main Methods:

• Analysis of the European Union's AI Act, specifically Articles 14 and 26.
• Conceptual framework development for AI governance in healthcare institutions.
• Comparative analysis of oversight models, including "Human-in-the-Loop" and AI Governance Committees.

Main Results:

• Significant legal and operational ambiguity exists in assigning ongoing AI oversight responsibility among providers, deployers, and clinicians.
• Procedural compliance alone is insufficient; systemic, institution-wide governance structures are necessary for informed and continuous oversight.
• A multidisciplinary AI Governance Committee model is proposed to balance legal duties with ethical responsibilities and professional autonomy.

Conclusions:

• Effective AI oversight in healthcare necessitates formal, institution-wide governance structures beyond simple procedural compliance.
• Hospitals must establish clear roles, policies, and training to empower human overseers with AI literacy and authority.
• Multidisciplinary AI Governance Committees are crucial for ensuring AI enhances clinical judgment and patient advocacy while meeting regulatory demands.