自主监督表示的轨迹顺序目标 使用变压器学习时间医疗保健数据:模型开发和评估研究研究
Ali Amirahmadi1, Farzaneh Etminani2,3, Jonas Björk4
1Center for Applied Intelligent Systems Research in Health, Halmstad University, Halmstad, Sweden.
JMIR medical informatics
|June 4, 2025
概括
TOO-BERT是一种新的深度学习模型,通过更好地捕获患者的时间数据来增强电子健康记录 (EHR) 分析. 这种方法改善了对心力衰竭和阿尔茨海默病等疾病的预测.
科学领域:
- 人工智能的人工智能
- 生物医学信息学 生物医学信息学
- 机器学习 机器学习
背景情况:
- 电子健康记录 (EHR) 通过深度学习为改善患者护理提供了巨大的潜力.
- 由于患者轨迹中的复杂时间关系,对顺序EHR数据的建模具有挑战性.
- 现有的转换器模型使用掩盖语言建模 (MLM) 捕捉上下文,但与时间动态扎.
研究的目的:
- 通过解决捕捉时间依赖性的局限性来增强EHR序列建模.
- 引入一种基于变压器的新型模型TOO-BERT,以更好地了解患者的病历.
主要方法:
- 开发了轨迹顺序目标BERT (TOO-BERT),一个变压器模型,将一个新的轨迹顺序目标 (TOO) 与MLM预培训集成在一起.
- TOO-BERT通过区分有序的和置的医疗事件序列来进行预训练,专注于经常同时发生的代码/访问.
- 在MIMIC-IV和马尔摩饮食和癌症队列 (MDC) 数据集上评估TOO-BERT,与传统方法和MLM预训练的变压器进行比较.
主要成果:
- 在这两组数据中,Too-BERT在预测心力衰竭 (HF),阿尔茨海默病 (AD) 和长期停留 (PLS) 方面显著优于现有方法.
- 在MDC数据集上实现了对HF和AD预测的AUC得分的改进 (例如,HF从67.7%到73.9%).
- 在高频预测方面表现强,即使在MIMIC-IV数据集上的微调数据有限.
结论:
- 将时间顺序目标集成到经过MLM预训练的模型中,可以有效地捕捉到EHR数据中的复杂时间关系.
- TOO-BERT通过代表患者轨迹中的复杂结构模式,提供了对疾病进展的更深入的见解.
- 该模型提供了对患者健康旅程和疾病发展的更细致的理解.
相关概念视频
Transformers
1.2K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.2K
Transformers with Off-Nominal Turns Ratios
214
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
214
Types Of Transformers
1.1K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.1K
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Energy Losses in Transformers
988
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
988

