Large Language Models for Individualized Psychoeducational Tools for Psychosis: A Cross-Sectional Study

Area of Science:

• Artificial Intelligence in Mental Health
• Clinical Psychology
• Psychiatric Care

Background:

• Psychoeducation is crucial for patients with psychosis, their caregivers, and relatives.
• Existing resources may not always meet the diverse needs of individuals seeking information about psychosis.
• Large language models (LLMs) present a potential new avenue for delivering psychoeducational content.

Purpose of the Study:

• To evaluate the quality of GPT-4 generated responses to common psychosis-related psychoeducational questions.
• Assess responses based on accuracy, clarity, inclusivity, completeness, and clinical utility.
• Determine the potential role of LLMs in supporting psychosis psychoeducation.

Main Methods:

• A qualitative evaluation design was used.
• GPT-4 generated responses to 20 psychosis-related questions developed by clinicians.
• Two experts independently assessed responses using a six-domain rubric, resolving discrepancies through consensus.

Main Results:

• GPT-4 responses were generally accurate (2.88/3), clear (2.93/3), complete (0.93/1), and clinically useful (4.35/5).
• Inclusivity scores were lower (2.30/3), and responses exhibited high reading complexity (Flesch-Kincaid Grade Level 15.59).
• Some responses lacked nuance for complex clinical scenarios.

Conclusions:

• GPT-4 shows a limited adjunctive role in psychosis psychoeducation within clinician-guided settings.
• Readability and clinical relevance were strengths, but complexity and inclusivity issues pose accessibility challenges.
• Further research is necessary before widespread clinical integration due to accuracy, safety, and implementation concerns.