Generative artificial intelligence versus clinicians: Who diagnoses multiple sclerosis faster and with greater...

Area of Science:

• Neuroimmunology
• Artificial Intelligence in Medicine
• Digital Health

Background:

• Generation Z (Gen Z) will constitute the majority of new multiple sclerosis (MS) diagnoses.
• Younger patients increasingly use generative AI for medical advice before specialist consultations.
• Gen Z's tech-savviness and preference for instant, cost-effective information drive AI tool adoption.

Purpose of the Study:

• To evaluate ChatGPT's ability to diagnose MS earlier than the established clinical timeline.
• To assess diagnostic accuracy variations based on patient age, sex, and race/ethnicity.

Main Methods:

• Retrospective analysis of MS patient timelines (ages 18-59).
• Simulation of clinical timelines using ChatGPT-3.5 (GPT-3.5) with actual and derivative patient data.
• Kaplan-Meier survival analysis and Wilcoxon tests for time-to-diagnosis comparisons.
• Logistic regression to assess diagnostic accuracy before and after MRI data inclusion.

Main Results:

• ChatGPT diagnosed MS significantly faster (0.08 years) than the clinical timeline (0.35 years) (p<0.0001).
• Pre-MRI data: Males had lower diagnostic accuracy than females (p=0.05).
• Post-MRI data: Gen Z showed higher accuracy (p=0.01), while males (p=0.009) and White subjects (p=0.0004) had reduced accuracy compared to females and non-White subjects, respectively.

Conclusions:

• Generative AI offers rapid information access but isn't designed for healthcare.
• Anticipated increase in Gen Z's use of AI for medical information.
• AI-generated responses may lack generalizability and exhibit biases across demographic groups.