Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Area of Science:

• Artificial Intelligence
• Psychology
• Human-Computer Interaction

Background:

• Therapeutic support demand outstrips professional capacity.
• Virtual agents offer scalable solutions for behavior change support.
• Adaptive conversational strategies are crucial for effective motivational interviewing (MI).

Purpose of the Study:

• To develop a virtual agent capable of real-time, adaptive MI sessions.
• To personalize therapeutic interactions through dynamic verbal and nonverbal adjustments.
• To enhance digitally mediated therapeutic interventions.

Main Methods:

• Embodied virtual agent using Greta 2.0 platform.
• Nonverbal behavior generation via MODIFF diffusion model, adapting to user expressions.
• Dialogue generation using a large language model (LLM) with a reinforcement learning-based MI dialogue manager.
• User studies for validation of facial expressions and dialogue manager.

Main Results:

• The virtual agent dynamically adapts verbal and nonverbal behaviors in real-time.
• MODIFF model successfully adapts nonverbal cues to user facial expressions and motivation levels.
• LLM-enhanced dialogue manager adapts to user profiles, validated through user testing.
• The platform facilitates real-time, multimodal MI dialogues.

Conclusions:

• The developed virtual agent provides a novel, open-source tool for personalized, adaptive MI.
• This technology can significantly augment therapeutic support and facilitate behavior change.
• Real-time multimodal interaction capabilities enhance digitally mediated therapy.