为了评估和构建医学的多功能大型语言模型.
Chaoyi Wu1,2, Pengcheng Qiu1,2, Jinxin Liu3
1Shanghai Jiao Tong University, Shanghai, China.
NPJ digital medicine
|January 26, 2025
概括
我们创建了MedS-Bench来评估临床大语言模型 (LLM) 和MedS-Ins,一个数据集来改进它们. 我们精心调整的模型显示了医疗任务的显著收益.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在各个领域都很有前途,但在临床应用中需要专门的评估.
- 现有的基准可能无法充分捕捉医学语言理解和推理的复杂性和细微差别.
研究的目的:
- 引入MedS-Bench,这是一个用于评估临床环境中的LLMs的新基准,涉及11个不同的任务.
- 开发MedS-Ins,一个大规模的指令调整数据集,以提高医学LLM的绩效.
- 建立一个动态的排行榜,跟踪医学LLM的进步.
主要方法:
- 在MedS-Bench上评估了九个领先的LLM,确定了复杂的临床任务中的绩效差距.
- 构建了MedS-Ins,一个数据集,包含来自58个医疗机构的500万个实例,涵盖了122个任务.
- 在使用MedS-Ins的开源医学LLM上进行了指令调整,创建了MMedIns-Llama 3.
主要成果:
- 大多数评估的LLM在MedS-Bench中处理复杂的临床任务方面存在局限性.
- 在MedS-Ins上微调的MMedIns-Llama 3模型在多个临床基准上实现了卓越的性能.
- MedS-Ins包含19,000条指令,为医学LLM培训提供丰富的资源.
结论:
- MedS-Bench为评估临床LLM能力提供了一个强大的框架.
- MedS-Ins显著提高了在医疗环境中LLM的表现,解决了当前的局限性.
- MedS-Ins的开放可访问性和MedS-Bench排名表旨在促进医疗AI的协作进步.
相关概念视频
02:50Combination Therapies and Personalized Medicine
4.8K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.8K
01:26Pharmacokinetic Models: Comparison and Selection Criterion
38
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
38
02:28Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
01:20Pharmacokinetic Models: Overview
570
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
570