Generation of Backward-Looking Complex Reflections for a Motivational Interviewing-Based Smoking Cessation Chatbot...

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI) in Healthcare
• Behavioral Science and Digital Therapeutics

Background:

• Motivational interviewing (MI) is effective for smoking cessation but faces accessibility challenges.
• A chatbot, MIBot, aims to provide MI sessions, focusing on reflective listening.
• Backward-looking complex reflections (BLCRs) are a key MI technique not yet implemented in MIBot.

Purpose of the Study:

• To develop and evaluate a method for generating BLCRs using large language models (LLMs).
• To enhance the MIBot chatbot's capability for delivering MI-based smoking cessation support.

Main Methods:

• Utilized GPT-4, an advanced LLM, with prompt engineering to generate BLCRs.
• Developed prompts based on conversational history to elicit specific backward-looking reflections.
• Tested the method on 150 reflections from 50 MIBot transcripts, with quality assessed by independent raters.

Main Results:

• 88% of the 150 generated BLCRs met acceptability criteria.
• High inter-rater agreement (80%-88%) indicates reliable quality assessment.
• The LLM approach demonstrated effectiveness in generating contextually relevant therapeutic reflections.

Conclusions:

• The developed method is suitable for generating BLCRs in MI-style conversations.
• An automated checker is recommended to filter out any remaining unacceptable reflections.
• Highlights the potential of advanced LLMs for creating personalized, therapeutic digital interventions.