Related Experiment Video
Updated: Jun 28, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large Language Models for Breast and Cervical Cancers Communication: Mixed Methods Evaluation Study Assessing
Agnik Saha1, Victoria Churchill2, Anny D Rodriguez3
1Department of Computer Science, Georgia State University, Atlanta, GA, United States.
JMIR Cancer
|June 26, 2026
Summary
Large language models (LLMs) show promise for cancer communication but have trade-offs. General-purpose LLMs excel in quality, while medical LLMs offer simplicity, impacting safety and accessibility.
Area of Science:
- Artificial Intelligence in Healthcare
- Natural Language Processing for Public Health
- Digital Health Communication
Background:
- Effective communication on breast and cervical cancers faces challenges from misinformation and language barriers.
- Large language models (LLMs) present opportunities for scalable health communication.
- The balance between quality, safety, and accessibility of general-purpose versus medical LLMs is not well understood.
Purpose of the Study:
- To develop a framework for evaluating LLMs in generating breast and cervical cancer information.
- To systematically assess LLM performance focusing on linguistic quality, safety, trustworthiness, and communication effectiveness.
- To compare general-purpose and medical-domain LLMs for cancer-related health communication.
Main Methods:
- A mixed-methods evaluation of 5 general-purpose and 3 medical LLMs using real-world cancer questions.
- Assessment of LLM outputs for linguistic quality (fluency, coherence, accuracy), safety (toxicity, bias, harm), and communication (readability, empathy, clarity).
- Utilized domain expert qualitative ratings and quantitative metrics with statistical analyses (Welch ANOVA, Games-Howell, Hedges g).
Main Results:
- General-purpose LLMs (Llama 3, Gemma) showed higher linguistic quality and effectiveness but potentially lower accessibility due to complexity.
- Medical LLMs (MedAlpaca, BioMistral) produced simpler content but had lower scores in safety and empathy, with increased hallucination, bias, and toxicity.
- A clear trade-off was observed between domain specialization and communication quality/safety.
Conclusions:
- LLMs offer potential for improving digital cancer communication, but specialization presents quality and safety trade-offs.
- Future health-focused LLMs should explore hybrid approaches to enhance trust, clarity, and clinical relevance.
- Addressing the balance between specialized knowledge and accessible, safe communication is crucial for patient-facing tools.
Related Concept Videos
01:21Cancer Survival Analysis
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
01:21Barriers to Effective Communication II
The barriers to effective communication also include cultural barriers, semantic barriers, gender barriers, and time constraints.
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...