提高法国临床笔记分类的变压器性能,使用有限数据集上的专家混合
Thanh-Dung Le1,2, Philippe Jouvet3, Rita Noumeir1
1Biomedical Information Processing Laboratory, École de Technologie SupérieureUniversity of Quebec Quebec City QC G1K 9H6 Canada.
概括
一个新的专家混合 (MoE) 变压器模型有效地分类小的法国临床文本. 这种NLP方法为数据和计算资源有限的医院提供了更快,更可行的替代方案.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 机器学习 机器学习
- 临床信息学 临床信息学
背景情况:
- 变压器模型在NLP中表现出色,但由于计算需求,在小规模的临床文本分类方面遇到了困难.
- 现有的生物医学预训练模型 (例如CamemBERT-bio,DrBERT) 是计算密集型的,限制了它们在医院环境中的使用.
- 需要在资源有限的医院环境中为法国临床叙述提供高效的NLP解决方案.
研究的目的:
- 开发和评估一个定制的专家混合 (MoE) 变压器模型,用于分类小规模的法国临床文本.
- 为解决内部医院应用有限数据和低资源计算的挑战.
- 在资源有限的环境中为临床文本分析提供实用的NLP替代方案.
主要方法:
- 开发一个定制的MoE变压器架构.
- 培训和评估来自CHU Sainte-Justine医院的法国小型临床文本.
- 对DistillBERT,CamemBERT,FlauBERT和标准变压器模型进行比较分析.
主要成果:
- 能源部变压器实现了87%的准确度,87%的精度,85%的回忆率和86%的F1分数.
- 在同一数据集上表现优于DistillBERT,CamemBERT,FlauBERT和变压器模型.
- 实现的训练速度至少比可比的生物医学BERT模型快190倍,尽管性能略低.
结论:
- 定制的MoE-Transformer提供了一个计算效率高,有效的解决方案,用于分类小的法语临床文本.
- 在有限的临床环境中,为高资源的生物医学BERT模型提供了可行的替代方案.
- 显示了将其纳入临床决策支持系统的巨大潜力,特别是在儿科重症监护病房.
相关概念视频
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Aggregates Classification
386
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
386
Classification of Neurotransmitters
3.6K
Neurotransmitters play a crucial role in the communication between neurons in the autonomic nervous system. Neurons in the autonomic nervous system can be cholinergic or adrenergic depending on the neurotransmitters synthesized. Cholinergic neurons use acetylcholine as their primary neurotransmitter. This includes all the preganglionic fibers of the sympathetic and pre- and postganglionic fibers of the parasympathetic nervous systems. In addition, neurons of the somatic nervous system also use...
3.6K
Transformers with Off-Nominal Turns Ratios
205
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
205
Classification of Signals
896
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
896
Three-Winding Transformers
313
Three identical single-phase transformers can be configured to form a three-phase transformer connection, which involves high-voltage and low-voltage windings. The high-voltage windings are denoted by capital letters A-B-C, while the low-voltage windings are labeled with lowercase letters a-b-c, representing their respective phases. This notation helps distinguish between the high and low voltage sides of the transformer.
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
313


