克莱夫特:语言图像对比学习,使用高效的大型语言模型和快速微调.
Yuexi Du1, Brian Chang1, Nicha C Dvornek1,2
1Department of Biomedical Engineering, Yale University, New Haven, CT, USA.
概括
我们开发了CLEFT,这是一种有效的对比学习方法,用于医学成像. 它通过学习基于上下文的提示来提高性能,减少模型大小和培训需求.
科学领域:
- 人工智能的人工智能
- 医学成像分析 医学成像分析
- 计算机视觉 计算机视觉
背景情况:
- 对比性语言图像预训练 (CLIP) 在自我监督学习方面表现有前途,但需要大量的计算资源和大型数据集,限制了其医学应用.
- 目前的CLIP方法通常依赖于从图像标签中手动获得的提示,可能缺少细微的临床信息.
- 医疗数据集往往更小,更难以获取,这对资源密集型模型构成了挑战.
研究的目的:
- 为医疗应用引入一种高效的语言图像对比学习方法 (CLEFT).
- 制定学习基于背景的提示策略,以更好地利用临床数据.
- 提高医疗成像中自我监督表示学习的性能和效率.
主要方法:
- 开发了CLEFT,一种新的对比学习框架,集成高效的大型语言模型和快速微调.
- 实施了一种有效的策略,从临床诊断数据中学习基于上下文的提示.
- 利用广泛的预先训练的语言和视觉模型来增强表现学习.
主要成果:
- 在多个胸部X射线和乳房扫描数据集上实现了最先进的性能.
- 在可训练模型大小 (39%) 和语言模型参数 (到BERT编码器的4%) 中显著减少.
- 展示了基于上下文的提示在弥合临床数据和简单标签之间的差距方面的有效性.
结论:
- 在医学成像方面,CLEFT为现有的CLIP类方法提供了一个参数高效和高性能的替代方案.
- 提议的快速学习策略增强了临床数据对自我监督学习的有用性.
- 由于CLEFT的效率,它适用于数据和计算资源有限的医疗应用.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.8K
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
1.4K
相关概念视频
Language and Cognition
321
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
321
Improving Translational Accuracy
2.5K
2.5K
Language Development
317
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
317
