用葡萄牙语评估医疗问题答案的大型语言模型:开源与封闭源方法对比
1Imaging Department, Unidade Local de Saúde de Trás-os-Montes e Alto Douro, Vila Real, PRT.
Cureus
|June 16, 2025
概括
像GPT-4o这样的闭源大语言模型 (LLM) 在医学知识评估中显示出比LLaMA 3.1.这样的开源模型更高的准确性. 促销策略提供了各种各样的好处,GPT-4o在整体上表现出色.
科学领域:
- 医疗教育 技术 技术 医学教育
- 医疗保健中的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 越来越多地被用于评估医学知识.
- 评估不同LLM架构的性能对于它们在高风险医疗评估中的应用至关重要.
- 葡萄牙国家居留权准入考试是医学知识评估的基准.
研究的目的:
- 在医学知识评估中,将一个封闭源代码的LLM (GPT-4o) 与一个开源的LLM (LLaMA 3.1) 进行比较.
- 评估各种提示策略对医学背景下LLM绩效的影响.
- 评估LLM对医学问题的答案中的潜在偏见.
主要方法:
- 来自2023年葡萄牙国家居留权入学考试的148个多项选择题被使用.
- 使用五种不同的提示策略测试了GPT-4o和LLaMA 3.1.
- 分析了模型准确性,跨临床领域的性能和响应偏差.
主要成果:
- 在所有提示方法中,GPT-4o 始终实现了比 LLaMA 3.1 高出 7-11% 的精度.
- 连锁思维提示显示GPT-4o的最大准确性,但不显著.
- 这两种模型在儿科表现最好,在手术和精神病学表现最不准确;LLaMA 3.1表现出依赖提示的偏差.
结论:
- 封闭源LLM目前为葡萄牙医疗知识评估提供了更高的准确性.
- 开源的LLM在数据控制方面提供了优势,但可能需要对特定领域进行微调以获得最佳性能.
- 对于高风险的医疗应用,建议对微调开源模型进行进一步的研究.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
595
05:56Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
Published on: April 14, 2023
2.7K
相关概念视频
Improving Translational Accuracy
2.7K
2.7K
Language and Cognition
466
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
466
Language Development
462
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
462
Classification of Illness
8.0K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.0K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
132
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
132
