ChatCLIDS: Simulating Persuasive AI Dialogues to Promote Closed-Loop Insulin Adoption in Type 1 Diabetes Care

Area of Science:

• Artificial Intelligence in Healthcare
• Behavioral Science
• Endocrinology

Background:

• Real-world adoption of closed-loop insulin delivery systems (CLIDS) for type 1 diabetes is hindered by behavioral, psychosocial, and social factors, not technical limitations.
• Effective behavior change interventions are crucial for improving CLIDS uptake and managing type 1 diabetes.
• Evaluating AI-driven persuasive dialogue for health behavior change requires robust and realistic simulation frameworks.

Purpose of the Study:

• To introduce ChatCLIDS, the first benchmark for evaluating LLM-driven persuasive dialogue in health behavior change.
• To simulate realistic multi-turn interactions between AI nurse agents and virtual patients with diverse adoption barriers.
• To assess the efficacy of LLMs in overcoming resistance and promoting CLIDS adoption in type 1 diabetes.

Main Methods:

• Developed ChatCLIDS, a framework with expert-validated virtual patients exhibiting heterogeneous profiles and adoption barriers.
• Simulated longitudinal counseling and adversarial social influence scenarios using nurse agents with evidence-based persuasive strategies.
• Evaluated the performance of various large language models (LLMs) in adapting strategies and influencing patient behavior over simulated interactions.

Main Results:

• While larger and more reflective LLMs demonstrated some adaptation of strategies over time, all models exhibited limitations in overcoming patient resistance.
• Realistic social pressure significantly challenged the persuasive capabilities of current LLMs.
• The ChatCLIDS benchmark provided a high-fidelity, scalable platform for multi-dimensional evaluation of AI in health behavior change.

Conclusions:

• Current LLMs show critical limitations in effectively driving health behavior change, particularly in overcoming resistance and social influence.
• ChatCLIDS offers a valuable testbed for advancing trustworthy persuasive AI in healthcare, enabling rigorous evaluation of AI-driven interventions.
• Further research is needed to enhance LLM capabilities for nuanced and effective health behavior modification in complex real-world scenarios.