对生物医学自然语言处理的联合学习进行深入评估,以提取信息
Le Peng1, Gaoxiang Luo2, Sicheng Zhou3
1Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN, USA.
NPJ digital medicine
|May 15, 2024
概括
联合学习 (FL) 通过实现协作培训,同时保持数据隐私,从而增强生物医学自然语言处理 (NLP). FL模型的表现优于单个客户端模型和少数镜头提示的大型语言模型 (LLM).
科学领域:
- 生物医学自然语言处理 (NLP)
- 机器学习 机器学习
- 数据 隐私 数据 隐私 数据
背景情况:
- 语言模型 (LMs) 已经改变了NLP,但医疗数据隐私法规 (HIPAA,GDPR) 限制了LM培训.
- 联合学习 (FL) 为协作模式培训提供了一种保护隐私的去中心化方法.
研究的目的:
- 评估FL在培训LM对生物医学NLP任务方面的有效性.
- 将FL模型的性能与传统培训方法和预先培训的LLMs进行比较.
主要方法:
- 在8个体中,FL应用于2个生物医学NLP任务,使用6个LM.
- 通过比较FL模型与个别训练的模型,中央调查的数据模型和少数镜头提示的LLMs来评估性能.
主要成果:
- FL模型的表现始终优于在单独的客户数据上训练的模型.
- 在某些情况下,FL模型的性能与中央训练模型相美.
- 增加FL的客户数量降低了业绩,尽管基于变压器的模型显示了弹性.
- FL模型显著优于预先训练有素的LLM,使用了几次射击提示.
结论:
- 联合学习是培训生物医学领域的LM的可行和有效策略,解决了数据隐私问题.
- FL为传统的培训方法提供了有竞争力的替代方案,并且在特定的生物医学NLP任务中表现出高于少数射击提示的LLM的优越性能.
相关概念视频
01:11Genetic Lingo
Overview
01:25Introduction to Language of Pathophysiology l
Pathophysiology investigates how biological mechanisms—typically starting at the cellular level—disrupt normal bodily functions. It bridges anatomy and physiology to explain the progression of disease. With this foundation, it is important to understand the following key terms used to describe disease processes: Diagnosis:The process of identifying a disease using clinical evaluation, including signs (objective evidence like rashes), symptoms (subjective experiences like pain), laboratory test...
01:17Introduction to Language of Pathophysiology ll
This lesson explores key terms that describe how diseases progress, their outcomes, and their distribution in populations.Diagnostic tests identify diseases and monitor treatment. These include blood and urine tests, biopsies, imaging (X-ray, MRI), and detection of infectious agents.Remission is a reduction or disappearance of symptoms.Exacerbation refers to the worsening of symptoms, such as increased wheezing during an asthma attack.A precipitating factor triggers an acute episode, while a...