The Unexpected Harms of Artificial Intelligence in Healthcare: Reflections on Four Real-World Cases

Rapid advances in Artificial Intelligence (AI), especially with large language models, present both opportunities and challenges in healthcare. This article analyzes real-world AI-related harms in healthcare.

We selected four recent AI-related incidents from the AIAAIC Repository.

The incidents discussed include: Whisper's harmful hallucinations; UNOS's algorithm delaying transplants for black patients; the WHO's S.A.R.A.H. chatbot providing inaccurate health information; and Character AI's chatbot promoting disordered eating among teens.

These incidents highlight diverse risks, from misinformation to safety concerns, involving both industry and institutional providers. The article emphasizes the need for systematic reporting of AI-related harms, concerns about security, privacy, and ethics, and calls for a centralized health-specific database to enhance patient safety and understanding.