Emerging shadow health systems: regulating health-focused generative AI chatbots from a European perspective

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Public Health Policy

Background:

• Generative AI chatbots, like ChatGPT Health, are becoming primary contact points for health information.
• These AI systems can influence patient behavior, healthcare capacity, and trust in medical expertise.
• Their growing use signifies the rise of 'shadow health systems' operating outside traditional healthcare regulations.

Purpose of the Study:

• To examine the public health implications of health-focused generative AI chatbots in Europe.
• To analyze the current regulatory landscape for these AI tools within the EU.
• To propose reforms addressing governance gaps in AI healthcare regulation.

Main Methods:

• Policy analysis of existing EU legal frameworks.
• Examination of real-world effects versus declared medical purposes of AI chatbots.
• Review of public health impacts and equity considerations.

Main Results:

• Current EU law inadequately regulates AI health tools, focusing on declared purpose rather than actual impact.
• Shadow health systems, governed privately, lack healthcare-specific safeguards.
• Existing regulations create significant gaps in legal protection and oversight.

Conclusions:

• Health-focused generative AI chatbots necessitate a regulatory shift from declared purpose to real-world effects.
• Urgent reforms are needed to establish appropriate safeguards for these emerging shadow health systems.
• Addressing these governance gaps is crucial for public health and health equity in Europe.