Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jul 4, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Controlled Benchmark Evaluation of a Geographically Grounded Digital Health Chatbot Against a General-Purpose Model.

Zhaoqiang Zhou1, John Geracitano1, Saif Khairat1

  • 1University of North Carolina at Chapel Hill, NC, USA.

Studies in Health Technology and Informatics
|July 3, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Association of primary care telehealth with preventable healthcare utilization and care continuity among patients with chronic conditions.

NPJ digital medicine·2026
Same author

A Community-Level Digital Health Readiness Index for the US.

JAMA network open·2026
Same author

Virtual nursing implementation and 30-day patient readmissions: a multi-site study.

NPJ digital medicine·2026
Same author

How Good Are Patient-Facing LLMs at Survivorship Questions? A Comparative User Evaluation Across Four Chatbots.

Studies in health technology and informatics·2026
Same author

Barriers to Patient Inclusivity in GeoHealth Platforms: Usability Testing Insights.

Studies in health technology and informatics·2026
Same author

A Tool for Virtual Nursing Utilization in Emergency Reception: An Exploratory Machine Learning Approach.

Studies in health technology and informatics·2026

A specialized Digital Health Index (DHI) chatbot, built on LLaMA, outperformed general AI like GPT-5.3 in delivering health information. The DHI chatbot showed superior geographic relevance, evidence transparency, and accuracy.

Area of Science:

  • Digital Health
  • Artificial Intelligence
  • Health Informatics

Background:

  • Large language models (LLMs) are emerging tools for health information delivery.
  • The effectiveness of LLMs for geographically specific digital health services is not well understood.
  • Domain-specific AI models may offer advantages over general-purpose ones in specialized fields.

Purpose of the Study:

  • To evaluate a domain-specific, LLaMA-based Digital Health Index (DHI) chatbot against a general-purpose model (GPT-5.3).
  • To assess the chatbots' performance in providing geographically grounded digital health information.
  • To determine the impact of domain specialization on AI accuracy, transparency, and local relevance.

Main Methods:

  • Development of a LLaMA-based DHI chatbot trained on digital health datasets.
Keywords:
Artificial IntelligenceDigital HealthGeoHealthHealth Informatics

Related Experiment Videos

Last Updated: Jul 4, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

  • Controlled benchmark evaluation using 15 items across five scenario types and three difficulty levels.
  • Independent expert rating of chatbot responses using a multidimensional 5-point Likert scale.

    • Main Results:

    • The DHI chatbot significantly outperformed GPT-5.3 across all six evaluated domains.
    • The most substantial improvements were observed in geographic handling, evidence transparency, and accuracy.
    • These performance advantages were statistically significant and consistent across varying difficulty levels.

    Conclusions:

    • Domain-specific conversational AI, like the DHI chatbot, shows greater potential for digital health information services.
    • Geographically grounded and specialized AI can provide more accurate, transparent, and locally relevant health information.
    • Future development should focus on domain-specific AI for enhanced digital health applications.