Evaluating user experiences with an AI chatbot for health-related social needs: A cross-sectional mixed methods study
Related Experiment Video
Updated: Nov 1, 2026
Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
Published on: December 23, 2025
Emre Sezgin1,2, Daniel I Jackson1, Syed-Amad Hussain1,3
1Center for Biobehavioral Health, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, United States of America.
Abstract:
Unmet health-related social needs (hrsns) significantly impact health. while screening tools exist, linking individuals to resources remains difficult. chatbots offer a potential solution for personalized, real-time navigation, but their usability and trustworthiness, particularly among high-need populations, need further study. we conducted a mixed-methods study with 128 caregivers to evaluate an ai chatbot, daphne, designed to identify hrsns and recommend resources. data on usability, task load, satisfaction, and trust were collected, along with qualitative feedback and conversation transcripts. regression and pairwise analyses explored associations between demographics, hrsns, and user experience. participants were mostly female (68%), 30-49 (71%), white (44%) or black/african american (36%), relied on medicaid/medicare (83%), and cared for a child with special healthcare needs (78%). they reported high usability (sus = 84.7), low task load (nasa-tlx = 6.8), high satisfaction (nps = 8.0), and high trust (mean = 4.1). nearly all (98%) reported unmet hrsns, including food insecurity (76%) and financial limitations (75%). chatbot sessions averaged 3 minutes/20 turns. daphne achieved 99% intent accuracy in 1,523 turns, focusing on financial, housing, and food needs. usability and trust were consistent across demographics, though higher education and lower income groups showed slight usability decreases. certain hrsns (transportation, utility disruptions) correlated with higher trust and satisfaction, suggesting particular value for users facing structural barriers. daphne demonstrated strong usability and trust across diverse groups. future designs should focus on longitudinal impact and effectiveness for improved accessibility and practical implementation. our study supports digital social care practices and provides empirical evidence that a chatbot can engage vulnerable populations to support unmet hrsns.