大型语言模型在多语言医学多选择题中的性能评估:混合方法研究研究
Livia Maria Strasser1, Wilma Anschuetz2, Fabio Dennstädt1,3
1Medical Knowledge and Decision Support, School of Medicine, University of St.Gallen, St.Jakobstrasse 21, St.Gallen, 9000, Switzerland, +41 71 224 32 00.
JMIR medical education
|March 11, 2026
概括
大型语言模型 (LLM) 在跨语言的医学问题上显示了不同的准确性,其中德语表现最好. 在英语中推动通常会改善结果,但人类监督对于可靠地融入医学教育至关重要.
科学领域:
- 医学教育 医学教育
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 人工智能 (AI) 正在改变医疗保健和医学教育.
- 大型语言模型 (LLM) 在医疗执照考试中展示了潜力.
- 根据语言,LLM的表现有所不同,因此需要进行跨语言的比较.
研究的目的:
- 评估法学士在德国,法国和意大利的医学多选择题上的表现.
- 在多语言医学教育中定量和定性地评估LLM能力.
- 在不同的语言背景下确定影响LLM准确性的因素.
主要方法:
- 混合方法研究分析了114个德语,法语和意大利语的多选择题.
- 对多个LLM (OpenAI,MetaAI,Anthropic,DeepSeek) 的定量性能分析.
- 对高绩效的LLM (GPT4o,Claude-Sonnet-3.7) 对不正确答案的答案解释的定性分析.
主要成果:
- 根据模型和语言,LLM准确性差异很大 (64%-87%),德国问题表现最好.
- 英语提示通常优于语言匹配提示,尽管顶级模型显示了可比的结果.
- 定性分析揭示了LLM解释中的推理错误,并确定了3个不准确的问题.
结论:
- 医学考试中的LLM成绩受模型,提示符和输入语言的影响,需要仔细选择.
- 通过LLM生成的解释可以提高医疗问题的质量,这取决于数据安全性.
- 人类监督对于细微的医疗内容至关重要,并且需要持续评估才能实现可靠的LLM整合.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.8K
05:56Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
Published on: April 14, 2023
3.3K
相关概念视频
01:27Language and Cognition
918
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
918
02:07Improving Translational Accuracy
15.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.3K
02:07Improving Translational Accuracy
3.7K
3.7K