Disparities in seizure outcomes revealed by large language models

Area of Science:

• Medical Informatics
• Neurology
• Health Services Research

Background:

• Large-language models (LLMs) offer potential for healthcare transformation but may perpetuate or introduce biases.
• Social determinants of health influence epilepsy care access, yet their effect on seizure outcomes among those with access is not well understood.

Purpose of the Study:

• To evaluate an epilepsy-specific LLM for intrinsic bias across demographic groups.
• To determine if demographic factors are associated with different seizure outcomes using LLM-extracted data.

Main Methods:

• An epilepsy-specific LLM was assessed for prediction accuracy and confidence across race, ethnicity, sex, income, and insurance status.
• LLM-classified seizure freedom from 84,675 clinic visits (25,612 patients) was analyzed using univariable and multivariable models to identify outcome disparities.

Main Results:

• The LLM demonstrated minimal bias in prediction accuracy or confidence across demographic groups.
• Multivariable analysis revealed worse seizure outcomes for females (OR 1.33), publicly insured patients (OR 1.53), and individuals from lower-income zip codes (OR ≥1.22).
• Black patients showed worse outcomes in univariable but not multivariable analysis compared to White patients.

Conclusions:

• The epilepsy-specific LLM exhibited no significant intrinsic bias against demographic groups.
• LLM analysis identified significant disparities in seizure outcomes linked to sex, insurance status, and socioeconomic factors.
• These findings underscore the urgent need to address and reduce health disparities in epilepsy care.