PMC-LLaMA:朝着建立医学开源语言模型的方向
Chaoyi Wu1,2, Weixiong Lin1,2, Xiaoman Zhang1,2
1Cooperative Medianet Innovation Center (CMIC), Shanghai Jiao Tong University, Shanghai, 200240, China.
概括
我们开发了PMC-LLaMA,这是一个开源的医疗大语言模型 (LLM),基于广泛的生物医学数据进行训练. 这种强大而轻量级的模型在医疗问答任务中表现优于现有系统.
科学领域:
- 人工智能的人工智能
- 生物医学信息学 生物医学信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 是有前途的,但缺乏针对医疗应用的特定领域的知识.
- 一般的LLM在医学等专业领域的精度很难达到.
- 需要开源,医疗调整的法学士.
研究的目的:
- 开发PMC-LLaMA,一个强大的,开源的大型语言模型,专为医疗应用量身定制.
- 通过对特定领域的知识注入和微调来提高医学领域的LLM绩效.
- 为医学研究提供基础性的,可训练的生成语言骨干.
主要方法:
- 通过以数据为中心的知识注入,调整了一个通用目的的LLM.
- 整合了480万篇生物医学学术论文和3万本医学教科书.
- 在医疗质量保证,推理和对话方面进行了特定领域的指令微调,使用了2.02亿个令牌.
主要成果:
- 轻量级的PMC-LLaMA (13B参数) 在医疗质量保证基准上表现优异.
- 在评估的医疗问答任务中,PMC-LLaMA超过了ChatGPT.
- 所有模型,代码和数据集将向研究社区发布.
结论:
- PMC-LLaMA是一个专门为医疗领域构建的开源LLM.
- 废弃性研究证实了医学LLM培训数据和模型规模的有效性.
- 一个大规模的,全面的指令调整数据集被贡献给研究界.
更多相关视频
09:37Navigating MARRVEL, a Web-Based Tool that Integrates Human Genomics and Model Organism Genetics Information
Published on: August 15, 2019
9.7K
13:21Comprehensive DNA Methylation Analysis Using a Methyl-CpG-binding Domain Capture-based Method in Chronic Lymphocytic Leukemia Patients
Published on: June 16, 2017
10.0K
相关概念视频
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
53
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
53
02:50Combination Therapies and Personalized Medicine
4.9K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.9K