Me-LLaMA:医学应用的大型语言模型基金会
Qianqian Xie1, Qingyu Chen1, Aokun Chen2
1Section of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, USA.
Research square
|June 3, 2024
概括
与现有的开源模型相比,Me-LLaMA是一种新的医疗大语言模型 (LLM) 家族,在医疗任务上表现出优越的性能. 这些模型通过利用专门的医疗数据和减轻灾难性遗忘来解决临床环境中的局限性.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在医学上表现有前途,但缺乏专门的医疗培训.
- 现有的开源医学LLM在临床应用方面存在局限性.
- 挑战在于将一般的LLM适应细微的医疗数据.
研究的目的:
- 介绍Me-LLaMA,一个新的医学大语言模型 (LLM) 家族.
- 使用LLaMA2架构和广泛的医疗数据集开发基础和聊天增强模型.
- 评估Me-LLaMA的表现与现有模型和基准相比.
主要方法:
- 在129B代币医疗数据集上利用了持续的预训练.
- 使用214k样本数据集调整员工指令.
- 在6个医疗任务和12个数据集中开发和使用医学指令基准评估 (MIBE).
主要成果:
- 在零射击,少数射击和监督学习方面,Me-LLaMA模型的表现优于现有的开源医疗LLM.
- 针对特定任务的指令调整的Me-LLaMA在7/8的MIBE数据集上超过了ChatGPT,在5/8的MIBE数据集上超过了GPT-4.
- Me-LLaMA证明了对灾难性遗忘问题的优越缓解.
结论:
- Me-LLaMA代表了开源医学LLM的重大进步,整合了生物医学和临床数据.
- 这些模型在一般和医疗任务上都提供了增强的性能,使它们适合医疗AI应用.
- Me-LLaMA为未来的医疗人工智能研究和部署提供了坚实的基础.
相关概念视频
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Language Development
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
Introduction to Language of Pathophysiology l
Pathophysiology investigates how biological mechanisms—typically starting at the cellular level—disrupt normal bodily functions. It bridges anatomy and physiology to explain the progression of disease. With this foundation, it is important to understand the following key terms used to describe disease processes: Diagnosis:The process of identifying a disease using clinical evaluation, including signs (objective evidence like rashes), symptoms (subjective experiences like pain), laboratory test...
Introduction to Language of Pathophysiology ll
This lesson explores key terms that describe how diseases progress, their outcomes, and their distribution in populations.Diagnostic tests identify diseases and monitor treatment. These include blood and urine tests, biopsies, imaging (X-ray, MRI), and detection of infectious agents.Remission is a reduction or disappearance of symptoms.Exacerbation refers to the worsening of symptoms, such as increased wheezing during an asthma attack.A precipitating factor triggers an acute episode, while a...


