对MIMIC-IV临床注释总结的大型语言模型进行基准测试
1Department of Biology, University of Southern Denmark, Campusvej 55, Odense, 5230 Denmark.
Journal of healthcare informatics research
|February 9, 2026
概括
这项研究将16个大型语言模型 (LLM) 作为临床注释总结的基准. 杰玛-3-27B在提取总结方面表现出色,而DeepSeek-R1-70B,Qwen-3-32B和GPT-4o在抽象总结方面领先.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 对医疗保健应用有希望.
- 临床笔记总结LLMs的有效性尚未得到充分研究.
- 对此任务的LLMs缺乏系统的比较.
研究的目的:
- 为了对临床笔记摘要的16个生成LLM进行基准测试.
- 评估抽象和抽象的总结方法.
- 评估性能,处理时间,成本和部署可行性.
主要方法:
- 在MIMIC-IV-Note数据集上对16个生成的LLM (OpenAI GPT,DeepSeek,Meta LLaMA,Google Gemma,Mistral Mixtral,阿里巴巴Qwen) 进行比较.
- 实施和评估提取和抽象的总结.
- 使用词汇 (ROUGE,BLEU,METEOR) 和语义 (COMET,BERTScore) 的指标.
- 评估处理时间,成本和部署可行性.
主要成果:
- 杰玛-3-27B在提取总结方面表现出卓越的性能.
- 在抽象总结方面,DeepSeek-R1-70B,Qwen-3-32B和GPT-4o是表现最好的.
- 较小的模型 (LLaMa-3-8B,Gemma-2-9B) 提供了具有竞争力的结果,提高了效率.
结论:
- 用于临床笔记总结的模型选择涉及性能,效率和部署环境之间的权衡.
- LLM的性能不仅仅取决于参数大小.
- 调查结果为将LLM整合到医疗保健工作流程中提供了实际见解.
相关概念视频
Language
919
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
919
Two-Compartment Open Model: IV Infusion
604
A two-compartment model is a vital tool in pharmacokinetics, providing an essential understanding of drug behavior, especially for those administered via zero-order intravenous infusion. This model outlines two compartments: the central compartment, where elimination occurs, and the peripheral compartment.
The model illustrates the decrease in plasma drug concentration from the central compartment with a specific equation. It shows that under steady-state conditions, the drug's input rate...
The model illustrates the decrease in plasma drug concentration from the central compartment with a specific equation. It shows that under steady-state conditions, the drug's input rate...
604
Components of Language
825
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
825
Language Development
925
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
925
Language and Cognition
806
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
806
One-Compartment Model: IV Infusion
524
Intravenous (IV) infusion is often utilized when continuous and controlled drug delivery is necessary, such as during surgery or in the treatment of chronic diseases. This method offers numerous advantages, including immediate drug action, precise control over dosage, and bypassing the first-pass metabolism.
The one-compartment model for IV infusion uses mathematical equations to describe the rate of change in drug quantity in the body. At steady-state or infusion equilibrium, the drug input...
The one-compartment model for IV infusion uses mathematical equations to describe the rate of change in drug quantity in the body. At steady-state or infusion equilibrium, the drug input...
524


