Related Experiment Video
Updated: Sep 14, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
690
EYE-Llama, an in-domain large language model for ophthalmology
Tania Haghighi1,2, Sina Gholami1, Jared Todd Sokol3
1Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA.
Iscience
|July 23, 2025
Summary
Domain-specific training enhances large language models (LLMs) for ophthalmology. EYE-Llama, trained on specialized data, shows superior performance in medical question-answering tasks compared to general LLMs.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Informatics
Background:
- Large language models (LLMs) require domain-specific data for optimal performance in specialized fields like medicine.
- General-purpose LLMs may lack the nuanced understanding necessary for accurate clinical decision-making support and patient education.
Purpose of the Study:
- To introduce EYE-Llama, a novel LLM specifically pretrained on ophthalmology-focused datasets.
- To evaluate EYE-Llama's question-answering (Q&A) capabilities against other leading LLMs in the medical domain.
Main Methods:
- EYE-Llama was pretrained on a comprehensive corpus of ophthalmology literature, including PubMed abstracts, textbooks, and online articles.
- The model was further fine-tuned using diverse question-and-answer pairs relevant to ophthalmology.
- Performance was assessed using BERT score, BART score, and BLEU metrics on benchmarks like MedMCQA and PubMedQA, comparing against Llama 2, Llama 3, Meditron, ChatDoctor, and ChatGPT.
Main Results:
- EYE-Llama demonstrated superior performance across multiple evaluation metrics compared to general and other medical LLMs.
- It outperformed Llama 2, Meditron, and ChatDoctor on the MedMCQA benchmark.
- EYE-Llama achieved 0.96 accuracy on the PubMedQA benchmark, surpassing all tested models.
Conclusions:
- Domain-specific pretraining and fine-tuning significantly enhance the performance of LLMs for medical Q&A.
- Specialized models like EYE-Llama offer substantial value for improving clinical decision support and patient education in ophthalmology.
