Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: May 2, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Korean Medical Consultation With Open-Weight Large Language Models: Pilot Comparative Evaluation of

Saeyoun Choi1, Donghyun Kim2,3, Ji-Hwan Jeon1

  • 1MAIN Corp, 1 Gangwon-daehak-gil, Room 1201 (Bodeum-gwan), Chuncheon City, Gangwon Province, 24341, Republic of Korea, 82 10-9840-2120.

JMIR Formative Research
|April 30, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Association Between the Use of Proton Pump Inhibitors and Osteoporosis/Fracture: Nested Case-Control Studies Using a National Health Screening Cohort.

Journal of clinical medicine·2026
Same author

Prevalence and Associated Factors of Depression Among Emergency Physicians in South Korea: Findings from the 2025 Korean Emergency Physician Survey.

Medicina (Kaunas, Lithuania)·2026
Same author

A Comprehensive Profiling System Integrating Myers-Briggs Type Indicator (MBTI) and Dominance, Influence, Steadiness, and Conscientiousness (DISC) for Personalized Health Training: Correlational Analysis and Usability Evaluation.

JMIR human factors·2025
Same author

Comparative Analysis of Blood Transfusion Accuracy and Hemolysis Rate of Transfusion Cartridge Set Between Conventional Infusion Pumps and Cylinder-Type Infusion Pumps.

Biomedicines·2024
Same author

Substantiation and Effectiveness of Remote Monitoring System Based on IoMT Using Portable ECG Device.

Bioengineering (Basel, Switzerland)·2024
Same author

Prediction of Postnatal Growth Failure in Very Low Birth Weight Infants Using a Machine Learning Model.

Diagnostics (Basel, Switzerland)·2023
Same journal

Identifying Behavior Change Techniques for Digital Interventions Addressing Alcohol and Tobacco Co-Use: Findings From a Delphi Consensus Study.

JMIR formative research·2026
Same journal

Use of Electronic Patient Record Systems for Rapid Response to an MHRA Public Assessment Report: Retrospective Observational Study.

JMIR formative research·2026
Same journal

Digital Cognitive Behavioral Therapy for Older Adults With Symptoms of Depression: Feasibility Cohort Study.

JMIR formative research·2026
Same journal

Using Ecological Momentary Assessment to Document and Investigate Caregiver Practices Between Pediatric Therapy Sessions: Prospective Pilot Cohort Study.

JMIR formative research·2026
Same journal

Virtual Reality-Based Relaxation Training and Symptom Improvement Among Inpatients With Depressive Disorders: Retrospective Nonrandomized Comparative Study.

JMIR formative research·2026
Same journal

Exploring Informal Caregivers' Perception of the Olera Digital Caregiving Assistance Platform for Dementia Care: Mixed Methods Evaluation Study.

JMIR formative research·2026
See all related articles
This summary is machine-generated.

Metadata filtering significantly enhances retrieval-augmented generation (RAG) for Korean health chatbots, improving accuracy and safety. Simple RAG augmentation alone is insufficient for specialized medical domains.

Area of Science:

  • Artificial Intelligence
  • Natural Language Processing
  • Medical Informatics

Background:

  • Development of an open-source large language model (LLM) chatbot for Korean health consultations.
  • Implementation using retrieval-augmented generation (RAG) and metadata filtering.

Purpose of the Study:

  • Analyze and compare the performance of a RAG-based chatbot against other leading language models for Korean health consultations.
  • Evaluate the impact of metadata filtering on RAG performance.

Main Methods:

  • Constructed a 10.4 GB Korean medical document corpus.
  • Quantitatively compared 5 open-source LLMs (Qwen3:4B, Mistral:7B, Llama-3.1:8B, Gpt-Oss:20B, Gemma3:27B) in baseline, RAG-only, and RAG with metadata filtering configurations.
  • Assessed performance using a validation set of 226 questions, scoring accuracy, safety, and helpfulness.
Keywords:
Korean health careLLMRAGhealth chatbotlarge language modelmetadata filteringretrieval-augmented generation

More Related Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.9K

Related Experiment Videos

Last Updated: May 2, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.9K

Main Results:

  • RAG alone did not significantly improve performance; some models saw decreased scores.
  • RAG with metadata filtering yielded statistically significant performance increases (P<.05) for most models.
  • Mistral:7B and Gpt-Oss:20B showed notable score increases; Gpt-Oss:20B achieved the highest safety score.

Conclusions:

  • RAG effectiveness in specialized domains like Korean medical consultation hinges on metadata filtering for information quality.
  • Simple information augmentation is insufficient; RAG benefits are limited when intrinsic model knowledge is high.
  • Performance enhancement strategies must consider both retrieval quality and the model's existing capabilities.