医疗文本的分类基于歧视性预训练模型和快速调整
Yu Wang1, Yuan Wang2, Zhenwan Peng1
1School of Biomedical Engineering, Anhui Medical University, Hefei, China.
Digital health
|August 10, 2023
概括
这项研究引入了医疗文本分类的快速调整,提高了ERNIE-Health模型的性能. 这种方法比传统的微调方法提高了准确性和训练效率.
科学领域:
- 医学自然语言处理 医学自然语言处理
- 医疗保健中的人工智能
- 计算语言学 计算语言学
背景情况:
- 医疗文本分类对于组织健康信息至关重要.
- 当前的方法经常使用预先训练的语言模型的微调,这可能会增加复杂性.
- 快速调整提供了一种方法,通过重构任务,更有效地利用预先训练的模型.
研究的目的:
- 调查医疗文本分类快速调整的有效性.
- 使用ERNIE-Health模型应用提示调整,这是一个歧视性的预先训练的语言模型.
- 在医学文本分类中,评估提示调节的性能与传统微调的性能.
主要方法:
- 制定了一个针对ERNIE-Health的预培训任务量身定制的快速调整战略.
- 通过将文本分类转化为面具预测任务,利用多令牌选择方法.
- 用模板包装的原始文本,用模型培训的[UNK]令牌取代类别标签.
主要成果:
- 在KUAKE-Question意图分类 (0.866) 和CHiP-临床试验标准 (0.861) 数据集上取得了高准确性.
- 与微调方法相比,在训练过程中表现出更快的损失减少.
- 验证了提示调整作为医学文本分类的有效技术.
结论:
- 提示调整是医疗文本分类任务的一个有希望的方法.
- 这种方法提高了预先训练的语言模型在特定领域应用中的性能.
- 这些发现为推进医学NLP研究提供了宝贵的见解.
更多相关视频
04:09Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
Published on: October 10, 2018
8.3K
12:18A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
7.6K
相关概念视频
Classification of Illness
7.6K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.6K
Classification of Signals
532
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
532
Force Classification
1.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.3K
Classification of Leukocytes
2.0K
Leukocytes are classified into two groups based on the presence or absence of cytoplasmic granules. Granular leukocytes, which contain granules, belong to the myeloid lineage and are divided into three subtypes: neutrophils, eosinophils, and basophils. These cells are roughly spherical and characterized by the granules in their cytoplasm.
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
2.0K
Classification of Neurotransmitters
3.0K
Neurotransmitters play a crucial role in the communication between neurons in the autonomic nervous system. Neurons in the autonomic nervous system can be cholinergic or adrenergic depending on the neurotransmitters synthesized. Cholinergic neurons use acetylcholine as their primary neurotransmitter. This includes all the preganglionic fibers of the sympathetic and pre- and postganglionic fibers of the parasympathetic nervous systems. In addition, neurons of the somatic nervous system also use...
3.0K
Improving Translational Accuracy
2.6K
2.6K
