Evaluating the Clinical Decision-Making Accuracy of Artificial Intelligence in Common Geriatric Syndromes Using...

Area of Science:

• Geriatric Medicine
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• The application of artificial intelligence (AI) tools, including ChatGPT, in clinical decision support is growing.
• Uncertainty exists regarding AI's role in geriatric medicine due to complex multimorbidity and care planning needs.

Purpose of the Study:

• To evaluate the clinical accuracy, completeness, and guideline adherence of ChatGPT responses to common geriatric scenarios.
• Assessment used standardized vignettes and expert geriatrician review.

Main Methods:

• Seven standardized geriatric vignettes (polypharmacy, falls, dementia, delirium, frailty, advance care planning, urinary incontinence) were used.
• Five geriatricians evaluated ChatGPT (GPT-5) responses using a rubric assessing accuracy, completeness, guideline alignment, safety, and clarity.
• Inter-rater reliability was measured using Krippendorff's alpha.

Main Results:

• ChatGPT performed best in clarity (66/70) and safety (63/70), with lower scores for accuracy (59/70) and completeness (55/70).
• Guideline alignment was generally strong (61/70), with advance care planning scoring highest and urinary incontinence lowest.
• Key omissions, such as missing assessments or guideline tools, were noted.

Conclusions:

• ChatGPT demonstrates potential as a supportive tool in geriatrics, providing clear, safe, and guideline-aligned information.
• The tool lacks clinical depth and may miss critical elements in complex cases.
• AI in geriatrics requires cautious use, expert supervision, and should not replace independent clinical judgment.