用于临床自然语言处理的轻量级变压器
Omid Rohanian1,2, Mohammadmahdi Nouriborji2,3, Hannah Jauncey4
1Department of Engineering Science, University of Oxford, Oxford, UK.
概括
研究人员为自然语言处理 (NLP) 任务开发了高效,紧的临床变压器. 这些轻量级模型与像BioBERT这样的大型模型相匹配,在临床文本挖掘上表现优于其他紧型模型.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 计算语言学 计算语言学
- 生物医学信息学 生物医学信息学
背景情况:
- 专门的预训练语言模型在医学NLP中表现有前途.
- 像BioBERT和BioClinicalBERT这样的现有模型通常是资源密集型的.
- 知识的蒸使得能够创建更小,更高效的模型.
研究的目的:
- 开发用于临床文字处理的紧,高效的语言模型.
- 通过知识蒸和持续学习,创建轻量级的临床变压器.
- 在各种临床文本挖掘任务中评估模型性能.
主要方法:
- 利用知识蒸和持续学习来开发轻量级的临床变压器.
- 范围模型参数数量从数百万到数以万计.
- 对多个NLP任务的标准数据集进行了广泛的评估.
主要成果:
- 开发了高效,紧的临床变压器,其性能与较大的模型 (例如,BioBERT) 相提并论.
- 与其他在一般或生物医学数据上训练的紧型号相比,实现了更高的性能.
- 在各种临床文本挖掘任务中表现出有效性,包括NER和关系提取.
结论:
- 这项研究是第一个全面的努力,为临床NLP创建高效和紧的变压器.
- 开发的轻量级模型为资源有限的临床文本分析提供了可行的替代方案.
- 模型和代码是公开的,以促进可复制性和进一步的研究.
相关概念视频
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Clinical Trials: Overview
2.8K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.8K


