Empowering Informal Caregivers of Persons With Early-Stage Dementia by Large Language Models: Mixed Methods...

Area of Science:

• Gerontology
• Health Informatics
• Artificial Intelligence in Healthcare

Background:

• Informal caregivers need accessible knowledge and support for early-stage dementia care.
• Large language models (LLMs) offer potential for caregiving information but carry risks and challenges.
• Improving LLM-generated responses for dementia caregiving requires further exploration.

Purpose of the Study:

• Examine risks and challenges of using baseline ChatGPT-4o for dementia caregiving support.
• Assess how an enhanced ChatGPT-4o, with updated knowledge, can mitigate these issues.

Main Methods:

• Developed two ChatGPT-4o conditions: baseline (C1) and enhanced (C2) using prompt engineering.
• Generated 64 responses to 32 caregiver questions, evaluated by 12 experts on multiple criteria.
• Conducted expert interviews for qualitative insights into response differences and design opportunities.

Main Results:

• Enhanced ChatGPT-4o (C2) showed significant improvements in actionability, relevance, and satisfaction over baseline (C1).
• No significant differences were found in accuracy, intelligibility, trustworthiness, safety, or perceived harm.
• Qualitative analysis highlighted differences in detail, relevance, and actionability, with C2 being more detailed and actionable.

Conclusions:

• Both models provided reasonable and intelligible responses for dementia caregiving.
• The enhanced model offered more relevant, practical, and actionable guidance, leading to higher caregiver satisfaction.
• LLM enhancements show promise in improving dementia caregiving support resources.