ChatGPT-4o in delirium recognition: a pilot study

Area of Science:

• Geriatric Medicine
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• Delirium is a prevalent yet often underdiagnosed condition in elderly patients, linked to severe health consequences.
• The utility of large language models (LLMs) in recognizing complex geriatric syndromes like delirium requires investigation.

Purpose of the Study:

• To evaluate the performance and consistency of ChatGPT-4o in identifying delirium from geriatric case reports.
• To compare ChatGPT-4o's capabilities with DeepSeek-V2 for delirium detection.

Main Methods:

• Twenty PubMed-indexed geriatric case reports were transformed into structured vignettes with delirium references removed.
• ChatGPT-4o was assessed in three conditions to measure delirium identification accuracy and temporal consistency.
• Outputs were rated by a geriatrician and neurologist for clinical plausibility and decision-support value.

Main Results:

• ChatGPT-4o achieved 70% delirium identification in independent sessions and 90% in a consecutive run; DeepSeek-V2 identified 50%.
• Substantial agreement was observed between independent ChatGPT-4o sessions (κ=0.76), but low agreement in the consecutive run suggested priming.
• Expert clinician ratings were significantly higher for correctly identified cases.

Conclusions:

• ChatGPT-4o demonstrates potential for delirium recognition in geriatric settings, though its accuracy is influenced by context.
• LLMs can function as vigilance-enhancing tools for clinicians, but require expert supervision and further validation.