Related Experiment Videos
When algorithms speak first: The public health risk of consumer AI in ALS diagnosis
Stéphane Mathis1, Gwendal Le Masson1
1Department of Neurology, Muscle-Nerve Unit, ALS Reference Center, University Hospital (CHU) of Bordeaux (Pellegrin Hospital), Bordeaux, France.
Background:
Consumer AI platforms are increasingly used by patients to interpret medical reports, including ENMG results for ALS. While AI shows promise in controlled clinical settings (e.g., stroke imaging, melanoma detection), consumer-facing tools often provide overconfident, context-free diagnostic assertions (e.g., 'definitive evidence of ALS'), leading to premature and potentially harmful life-altering decisions.
Objective:
To highlight the clinical, ethical, and regulatory risks of unregulated AI in ALS diagnosis and propose actionable solutions.
Discussion:
We present a case of AI-mediated misdiagnosis, analyze the limitations of consumer-facing AI (lack of clinical context, longitudinal data, and specialist oversight), and discuss the "authority paradox" (patients trusting AI outputs over clinicians' nuanced assessments). We propose a structured 4-step clinical approach for managing AI-mediated self-diagnoses and urge regulators to classify such tools as high-risk under the EU AI Act.
Conclusion:
The uncritical adoption of consumer AI in ALS diagnosis represents a public health risk. Clinicians, regulators, and developers must collaborate to ensure AI serves patients safely and ethically.