从零到英雄:利用变压器用于生物医学命名实体识别在零和少射击环境中
Miloš Košprdić1, Nikola Prodanović1, Adela Ljajić1
1Institute for Artificial Intelligence Research and Development of Serbia, Fruškogorska 1, Novi Sad, 21000, Serbia.
Artificial intelligence in medicine
|August 28, 2024
概括
这项研究引入了一种用于生物医学命名实体识别 (NER) 的新方法,该方法可在最小或无标记数据的情况下显著提高性能. 该方法可以有效地识别新的生物医学实体,优于现有的模型.
科学领域:
- 生物医学信息学 生物医学信息学
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 在生物医学中,监督的命名实体识别 (NER) 需要广泛的注释数据集,这些数据集的创建是昂贵和耗时的.
- 新实体的再培训模式是低效的,阻碍了当前NER系统的适应性.
研究的目的:
- 为生物医学领域开发一种零和少数射击的NER方法.
- 为解决数据注释和模型再培训对新生物医学实体的挑战.
主要方法:
- 将多类令牌分类转换为二进制令牌分类.
- 在众多数据集和生物医学实体上预训练模型来学习语义关系.
- 使用了一个微调的基于PubMedBERT的模型.
主要成果:
- 在9个不同的生物医学实体中获得了35.44% (零射击),50.10% (一射击),69.94% (10射击) 和79.51% (100射击) 的F1平均得分.
- 与之前的基于变压器的方法相比,证明了更高的性能.
- 显示了与基于GPT3的模型具有显著较少参数的可比结果.
结论:
- 拟议的方法有效地识别了具有有限或没有例子的新生物医学实体.
- 该方法为生物医学NER提供了更有效和更可扩展的解决方案.
- 公开发布的模型和代码有助于进一步的研究和应用.
相关概念视频
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Transformers
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
Types Of Transformers
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
Transformers with Off-Nominal Turns Ratios
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the rated...


