Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jul 2, 2026

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

Iterative Multidisciplinary Development and Evaluation of a Patient-Facing SDoH Chatbot Using Synthetic Data

Anna M Maw1, Alexander Lupi1,2, Rachel Johnson-Koenke1

  • 1Department of Emergency Medicine, University of Colorado Anschutz Medical Campus, Aurora, US.

JMIR Formative Research
|June 30, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Comparison of survival outcomes for people with HR+/HER2- metastatic breast cancer who received palbociclib, ribociclib, or abemaciclib with an aromatase inhibitor: a plain language summary.

Future oncology (London, England)·2026
Same author

Treatment outcomes with palbociclib plus an aromatase inhibitor in patients with metastatic breast cancer who also have cardiovascular diseases: a plain language summary.

Future oncology (London, England)·2026
Same author

Research and Scholarly Methods: Using Artificial Intelligence.

Journal of the American College of Clinical Pharmacy : JACCP·2026
Same author

Mind the Performance Gap: Examining Dataset Shift During Prospective Validation.

Proceedings of machine learning research·2026
Same author

An Examination of Buprenorphine and Norbuprenorphine Concentrations in Inpatients Receiving Medications for Opioid Use Disorder.

The journal of applied laboratory medicine·2026
Same author

Real-world treatment durations, subsequent treatments, and switching of CDK4/6 inhibitors among patients with HR+/HER2- metastatic breast cancer.

The oncologist·2026

This study developed and evaluated a social determinants of health (SDoH) chatbot using synthetic data and case simulations. The approach refined chatbot performance and evaluation metrics before clinical deployment.

Area of Science:

  • Artificial Intelligence in Healthcare
  • Digital Health Interventions
  • Health Informatics

Background:

  • Inconsistent social determinants of health (SDoH) data collection impacts patient outcomes.
  • Large language model (LLM)-powered chatbots show potential for scalable SDoH data collection.
  • Lack of rigorous evaluation methods for patient-facing SDoH chatbots hinders clinical adoption.

Purpose of the Study:

  • To describe an efficient, iterative, multidisciplinary method for developing and evaluating a patient-facing SDoH chatbot.
  • To optimize chatbot performance and evaluation rubric using synthetic data and case simulation prior to clinical deployment.

Main Methods:

  • Adapted a 10-criterion evaluation rubric from healthcare AI frameworks.
  • Applied rubric to 27 synthetic SDoH clinical scenarios role-played by a clinical social worker.

Related Experiment Videos

Last Updated: Jul 2, 2026

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

  • Multidisciplinary expert team (social worker, nurse practitioner, physician) rated chatbot-patient interactions.
  • Utilized quantitative (percent agreement, Fleiss' κ) and qualitative analysis for iterative refinement.

    • Main Results:

    • Chatbot achieved high ratings for accurate interpretation (98%), communication quality (99%), and adaptive questioning (99%).
    • Lower performance observed in domain focus (51%), data capture completeness (59%), and safety (69%).
    • Qualitative feedback refined rubric definitions, particularly for 'safety,' focusing on emergency recognition.

    Conclusions:

    • A formative feasibility approach using synthetic case simulation effectively refines patient-facing SDoH chatbots and evaluation rubrics.
    • Future work will involve external raters, patient input, repeated testing, and prospective clinical evaluation.