Neural Conversational Agent for Weight Loss Counseling: Protocol for an Implementation and Feasibility Study

Obesity is a common, serious and costly chronic disease. Current clinical practice guidelines recommend that providers augment the longitudinal care of people living with obesity with consistent support for the development of self-efficacy and motivation to modify their lifestyle behaviors. Lifestyle behavior change aligns with the goals of motivational interviewing (MI), a client-centered yet directive counseling modality. However, training health care providers to be proficient in MI is expensive and time-consuming, resulting in a lack of trained counselors and limiting the widespread adoption of MI in clinical practice. Artificial intelligence (AI) counselors accessible via the internet can help circumvent these barriers.

The primary objective is to explore the feasibility of conducting unscripted MI-consistent counseling using Neural Agent for Obesity Motivational Interviewing (NAOMI), a large language model (LLM)-based web app for weight loss counseling. The secondary objectives are to test the acceptability and usability of NAOMI's counseling and examine its ability to shift motivational precursors in a sample of patients with overweight and obesity recruited from primary care clinics.

NAOMI will be developed based on recent advances in deep learning in four stages. In stages 1 and 2, NAOMI will be implemented using an open-source foundation LLM and (1) few-shot learning based on a prompt with task-specific instructions and (2) domain adaptation strategy based on fine-tuning LLM using a large corpus of general psychotherapy and MI treatment transcripts. In stages 3 and 4, we will refine the best of these 2 approaches. Each NAOMI version will be evaluated using a mixed methods approach in which 10 adults (18-65 years) meeting the criteria for overweight or obesity (25.0≥BMI≤39.9) interact with NAOMI and provide feedback. NAOMI's fidelity to the MI framework will be assessed using the Motivational Interviewing Treatment Integrity scale. Participants' general perceptions of AI conversational agents and NAOMI specifically will be assessed via Pre- and Post-Interaction Questionnaires. Motivational precursors, such as participants' confidence, importance, and readiness for changing lifestyle behaviors (eg, diet and activity), will be measured before and after the interaction, and 1 week later. A qualitative analysis of changes in the measures of perceptions of AI agents and counselors and motivational precursors will be performed. Participants will rate NAOMI's usability and empathic skills post interaction via questionnaire-based assessments along with providing feedback about their experience with NAOMI via a qualitative interview.

NAOMI (version 1.0) has been developed. Participant recruitment will commence in September 2024. Data collection activities are expected to conclude in May 2025.

If proven effective, LLM-based counseling agents can become a cost-effective approach for addressing the obesity epidemic at a public health level. They can also have a broad, transformative impact on the delivery of MI and other psychotherapeutic treatment modalities extending their reach and broadening access.

PRR1-10.2196/60361.