Related Experiment Video
Updated: Jul 9, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
594
A medical multimodal large language model for future pandemics.
Fenglin Liu1, Tingting Zhu2, Xian Wu3
1Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK. fenglin.liu@eng.ox.ac.uk.
NPJ Digital Medicine
|December 2, 2023
Summary
A new medical multimodal large language model (Med-MLLM) effectively learns from unlabeled data, enabling rapid adaptation for rare diseases like COVID-19 with minimal labels. This AI improves clinical decision-making across various data types and languages.
Area of Science:
- Artificial Intelligence in Medicine
- Medical Imaging Analysis
- Natural Language Processing in Healthcare
Background:
- Supervised deep neural networks require extensive labeled data, limiting their application to rare diseases.
- Physician workload and diagnostic efficiency can be improved with AI-assisted clinical decision-making.
- Unlabeled medical data represents a vast, underutilized resource for AI model training.
Purpose of the Study:
- To develop a medical multimodal large language model (Med-MLLM) capable of learning from unlabeled data.
- To enable rapid deployment and adaptation of AI models for rare diseases with limited labeled data.
- To create a versatile AI tool supporting clinical tasks involving both visual (radiographs) and textual (reports) medical data.
Main Methods:
- Developed a Med-MLLM for radiograph representation learning, integrating image understanding, text semantics, and clinical phenotypes from unlabeled data.
- Utilized multimodal data including chest X-rays, CT scans, medical reports, and clinical notes.
- Evaluated the model's performance on COVID-19 datasets in retrospective and prospective settings, across different variants, languages, and downstream tasks (reporting, diagnosis, prognosis).
Main Results:
- The Med-MLLM demonstrated effective learning of broad medical knowledge from unlabeled data.
- The model showed rapid adaptability to rare diseases, requiring only limited labels for deployment.
- Accurate and robust COVID-19 decision-support was achieved across diverse datasets, languages (English, Chinese, Spanish), and tasks, even with minimal labeled data.
Conclusions:
- Med-MLLM offers a powerful solution for leveraging unlabeled medical data to overcome data scarcity in rare disease diagnosis.
- The model's multimodal capabilities and adaptability enhance its utility for real-world clinical decision support systems.
- This approach holds significant promise for improving diagnostic accuracy and efficiency in the face of emerging and rare diseases.
Related Concept Videos
01:18Steps in Outbreak Investigation
133
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
133
02:28Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
01:27Language and Cognition
352
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
352