使用基于极端多标签长文本转换器模型的自动化ICD编码
Leibo Liu1, Oscar Perez-Concha1, Anthony Nguyen2
1Centre for Big Data Research in Health, University of New South Wales, Sydney, Australia.
Artificial intelligence in medicine
|October 2, 2023
概括
优化变压器模型实现了自动化国际疾病分类 (ICD) 编码的最新性能. 这些模型显著提高了复杂的医疗编码任务的准确性,超过了以前的基准.
科学领域:
- 自然语言处理自然语言处理.
- 医疗信息学 医疗信息学
- 机器学习 机器学习
背景情况:
- 变压器模型对自然语言处理任务有希望.
- 自动化国际疾病分类 (ICD) 编码带来了极端的标签集和长文本的挑战.
- 现有的变压器模型,如PLM-ICD和XR-Transformer正在探索ICD编码.
研究的目的:
- 调查和优化基于变压器的模型,用于自动化ICD编码.
- 为了应对极端标签集和ICD编码中长文本分类的挑战.
- 提出和评估一个基于变压器的新型模型,XR-LAT.
主要方法:
- 通过使用更长的序列长度进行训练,优化了最先进的PLM-ICD模型.
- 扩展了XR-Transformer模型,以支持ICD编码任务的更长的序列.
- 开发和训练了一种新型模型,XR-LAT,利用层次代码树,标签明智的注意力,知识转移和动态负样.
主要成果:
- 优化的PLM-ICD模型实现了新的最先进的微F1得分,在MIMIC-III上达到60.8%,在MIMIC-II上达到50.9%.
- 对于ICD编码,XR-Transformer在所有指标上表现不佳.
- 与之前的最先进技术相比,XR-LAT模型显示出具有竞争力的表现,宏观AUC提高了2.1% (MIMIC-III) 和5.1% (MIMIC-II).
结论:
- 优化的PLM-ICD模型代表了MIMIC-III和MIMIC-II数据集上的自动化ICD编码的最新技术.
- 新的XR-LAT模型提供了竞争力的性能,展示了复杂的ICD编码任务的潜力.
- 基于变压器的方法对于解决自动化ICD编码的挑战是有效的.
相关概念视频
Transformers with Off-Nominal Turns Ratios
173
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
173
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
The Ideal Transformer
419
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
419
Transformers in Distribution System
118
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
118
Improving Translational Accuracy
11.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.5K
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K


