缩小医疗证据总结的开源和商业大型语言模型之间的差距
Gongbo Zhang1, Qiao Jin2, Yiliang Zhou3
1Department of Biomedical Informatics, Columbia University, New York, NY, USA.
ArXiv
|October 7, 2024
概括
微调开源大语言模型 (LLM) 显著提高了它们总结医学证据的能力. 这种方法提高了透明度和性能,使它们与专有模式竞争.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 医疗信息学 医疗信息学
背景情况:
- 大型语言模型 (LLM) 显示了总结医学证据的潜力.
- 专有LLM提供高性能,但缺乏透明度,并产生供应商依赖.
- 开源LLM提供透明度和定制性,但通常表现不佳于专有模型.
研究的目的:
- 评估微调开源LLM用于医学证据总结的有效性.
- 确定微调是否可以弥合开源和专有LLMs之间的性能差距.
主要方法:
- 使用了MedReview基准数据集 (8,161个系统审查-摘要对).
- 精心调整了三个开源的LLM:PRIMERA,LongT5和Llama-2.这些LLM的编程已经完善.
- 使用ROUGE-L,METEOR和CHRF得分评估性能,以及人类和GPT-4模拟评估.
主要成果:
- 微调的LLM显示了显著的改善: +9.89红色L, +13.21METEOR, +15.82CHRF.
- 精心调整的LongT5在零射击设置下接近了GPT-3.5的性能.
- 更小的微调模型有时会超过更大的零射击模型.
- 在自动化和人类评估中,性能增长是一致的.
结论:
- 微调大大提高了开源LLM在医学证据总结方面的性能.
- 这种方法为专有模式提供了一个可行的,透明的替代方案.
- 结果指导选择LLM用于专业任务,如医学证据合成.
更多相关视频
相关概念视频
Clinical Trials: Overview
2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
Genomics
36.2K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
36.2K
Pharmacokinetic Models: Overview
607
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
607
Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
Bioequivalence: Overview
935
Pharmaceutical equivalents, by definition, are drug products with the same active ingredient in the same quantities, encapsulated in identical dosage forms, and intended for the same administration routes. These pharmaceutical equivalents are deemed bioequivalent if the bioavailability of the active entity in the drug preparations is similar. Moreover, pharmaceutical equivalents demonstrating bioequivalence are also regarded as therapeutically equivalent. This means that when used as directed,...
935
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K


