缩小开源和商业大型语言模型之间的差距,用于医学证据总结和总结
Gongbo Zhang1, Qiao Jin2, Yiliang Zhou3
1Department of Biomedical Informatics, Columbia University, New York, NY, USA.
NPJ digital medicine
|September 9, 2024
概括
微调开源大语言模型 (LLM) 显著提高了它们的医学证据总结能力. 即使是较小的微调模型也可以超过较大的专有模型,提高医疗AI的透明度和定制性.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 显示了总结医学证据的潜力.
- 专有LLM存在风险,例如缺乏透明度和依赖供应商.
- 开源LLM提供透明度,但通常表现不佳于专有模式.
研究的目的:
- 评估微调对医学证据总结的开源LLM性能的影响.
- 将微调的开源LLM与专有模型的性能进行比较.
主要方法:
- 使用MedReview数据集 (8161个系统审查-摘要对) 微调了三个开源的LLM (PRIMERA,LongT5,Llama-2).
- 通过人类评估和GPT-4模拟评估来评估性能.
- 将微调模型与零射击专有模型进行比较 (例如,GPT-3.5).
主要成果:
- 微调提高了所有测试的开源LLMs的性能.
- 精心调整的LongT5在零射击设置中实现了与GPT-3.5相比的性能.
- 在总结质量方面,更小的微调模型有时会超过更大的零射击模型.
结论:
- 微调是一种有效的策略,可以提高开源LLM医疗证据总结的性能.
- 开源的LLMs,当微调时,可以为专有模式提供一个具有竞争力和透明的替代方案.
- 这种方法提高了LLMs在医学研究和临床实践中的实用性.
更多相关视频
相关概念视频
Improving Translational Accuracy
2.5K
2.5K
Genomics
36.2K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
36.2K
Bioequivalence: Overview
944
Pharmaceutical equivalents, by definition, are drug products with the same active ingredient in the same quantities, encapsulated in identical dosage forms, and intended for the same administration routes. These pharmaceutical equivalents are deemed bioequivalent if the bioavailability of the active entity in the drug preparations is similar. Moreover, pharmaceutical equivalents demonstrating bioequivalence are also regarded as therapeutically equivalent. This means that when used as directed,...
944
Clinical Trials: Overview
2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
Pharmacokinetic Models: Overview
618
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
618
Targeted Cancer Therapies
7.5K
The targeted cancer therapies, also known as “molecular targeted therapies,” take advantage of the molecular and genetic differences between the cancer cells and the normal cells. It needs a thorough understanding of the cancer cells to develop drugs that can target specific molecular aspects that drive the growth, progression, and spread of cancer cells without affecting the growth and survival of other normal cells in the body.
There are several types of targeted therapies against...
There are several types of targeted therapies against...
7.5K


