Clinical decision support for bipolar depression using large language models.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Psychiatric Pharmacotherapy

Background:

• Managing bipolar depression is complex, with treatment guidelines offering some direction.
• Large language models (LLMs) show potential for aiding clinical decision-making.
• Current LLM applications in bipolar disorder pharmacotherapy require further investigation.

Purpose of the Study:

• To evaluate the efficacy of an LLM (GPT-4-turbo) in identifying optimal pharmacotherapies for bipolar depression.
• To compare the performance of an LLM augmented with treatment guidelines versus an un-augmented LLM.
• To assess the LLM's performance against expert recommendations and community clinician accuracy.

Main Methods:

• Fifty clinical vignettes of bipolar depression were created.
• Bipolar disorder experts identified optimal and contraindicated pharmacotherapies.
• An LLM (GPT-4-turbo) was tested with and without guideline augmentation on the vignettes.

Main Results:

• The augmented LLM identified expert-recommended optimal treatments in 50.8% of cases (Cohen's kappa = 0.31).
• The un-augmented LLM achieved 23.0% accuracy, significantly lower than the augmented model.
• Augmented LLM performance exceeded that of community clinicians (23.1% average accuracy).

Conclusions:

• LLMs prompted with evidence-based guidelines offer a scalable strategy for clinical decision support in bipolar depression.
• Further research is needed to validate efficacy and address potential overreliance and bias.
• Augmented LLMs show potential to improve pharmacotherapy selection accuracy for bipolar depression.