医学诊断的LLMs中的视觉文本集成:初步定量分析
Reem Agbareia1, Mahmud Omar2, Shelly Soffer3
1Ophthalmology Department, Hadassah Medical Center, Jerusalem, Israel.
Computational and structural biotechnology journal
|January 24, 2025
概括
多模式大语言模型 (LLM) 显示出高的诊断准确性,即使没有图像. 虽然添加视觉数据可以提高LLM和医生的表现,但医生从图像中获得更多,这表明需要加强LLM视觉处理.
科学领域:
- 医学诊断 医学诊断 医学诊断
- 医疗保健中的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 视觉数据对于医学诊断至关重要.
- 评估多式联运大型语言模型 (LLM) 用于整合文本和图像的诊断任务.
研究的目的:
- 评估GPT-4o和克劳德·索内特3.5.5的诊断性能.
- 与人类医生相比,用和没有集成视觉数据的LLM绩效进行比较.
主要方法:
- 使用了120个临床图片 (仅文本和图像).
- 图像来源于OPENi和NEJM挑战,确保它们对LLMs来说是新鲜的.
- 三名初级保健医生作为诊断准确性的基准.
主要成果:
- 在只有文本的场景中,LLM的表现优于医生的表现 (GPT-4o: 70.8%,Claude Sonnet 3.5: 59.5%,医生:39.5%).
- 图像集成改善了所有参与者的准确性,医生显示最大的收益 (医生:78.8%).
- 在提供图像的情况下,LLM在45-60%的案例中调整了他们的推理.
结论:
- 多式联络LLM表现出强大的诊断准确性,有时甚至在没有视觉输入的情况下超越医生.
- 虽然图像提高了LLM的表现,但相对收益低于人类医生观察到的.
- 需要在LLM视觉数据处理方面取得进一步的进步,以匹配医疗图像的类似人类的性能增长.
相关概念视频
Improving Translational Accuracy
8.6K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.6K
Classification of Illness
7.3K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.3K
Integrated Healthcare System
1.5K
An integrated healthcare system (IHS) is a set of organizations that provides for or arranges to provide coordinated and continuous service to a defined population. The IHS takes responsibility for that particular population's health status and outcome, both clinically and fiscally. An integrated healthcare system is a well-organized, well-coordinated, and collaborative network. The integrated delivery system is a network that connects different healthcare providers to deliver organized,...
1.5K
Health Literacy
4.0K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
4.0K
Documentation of Nursing Diagnosis
1.2K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.2K
Anatomical Terminology
11.8K
Knowledge of anatomy is essential to understand human biology and medicine. Anatomists and health care professionals use standard terminology to describe the human body with more precision and no ambiguity. Anatomical terms have mostly Greek and Latin-derived roots. Because these languages are rarely used in conversation, the meaning of words remains the same. Each term is made up of a root in between the prefixes and suffixes. The root of a term often refers to an organ, tissue, or condition,...
11.8K


