从临床文本中预测健康的行为决定因素,使用变压器模型和BiLSTM
Saad Althabiti1, Chuming Chen2, Cathy Wu3
1PhD Candidate, Center for Bioinformatics and Computational Biology, University of Delaware.
medRxiv : the preprint server for health sciences
|November 24, 2025
概括
我们的研究表明,类权重有效地处理不平衡的数据,以从临床笔记中预测健康行为决定因素 (BDoH). T5-EHR型号实现了最高的性能,超过了其他变压器型号.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 临床文字分析 临床文字分析
背景情况:
- 健康的社会和行为决定因素 (BDoH) 显著影响患者的结果.
- 在临床文本中,BDoH信息往往是无结构的,这限制了其实用性.
- 准确检测BDoH对于改善患者理解和决策至关重要.
研究的目的:
- 为了从医疗记录中提高BDoH的预测.
- 系统地比较基于变压器的模型 (Bio-ClinicalBERT,BioBERT,BioMedBERT,RoBERTa,T5-EHR) 使用和不使用BiLSTM.
- 解决阶级不平衡问题,并评估BDoH分类的生成与歧视模型.
主要方法:
- 在MIMIC-III数据集上评估了与BiLSTM结合的五种变压器模型.
- 对比过量抽样,不足抽样和对类不平衡的类权重.
- 评估独立模型性能和BiLSTM集成用于顺序建模.
- 与公布的结果进行基准测试,使用精度,回忆和F1分数.
主要成果:
- 在所有模型中,类权重在处理类不平衡方面被证明是最有效的.
- 整合BiLSTM提高了生物临床BERT,生物医疗BERT和生物BERT的性能.
- 作为独立模型,RoBERTa和T5-EHR的表现强,没有BiLSTM的好处.
- 在所有品牌中,T5-EHR获得了最高的F1分数,超过了之前的基线.
结论:
- 有效地处理阶级不平衡对于强大的BDoH预测至关重要.
- 通过捕捉顺序依赖,BiLSTM使特定模型受益.
- 罗伯塔和T5-EHR展示了他们预训练的表现的强度.
- 适应临床文本的T5-EHR模型为BDoH分类设定了新的最先进的状态.
相关概念视频
Models of Health Promotion and Illness Prevention I
2.6K
A model is a theoretical way to understand a concept or an idea. Models can overcome barriers to health regardless of diverse economic and cultural backgrounds. In addition, models make the task easier by providing different ways to approach complex issues. There are two major health promotion models: the health belief model and the health promotion model.
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
2.6K
Models of Health Promotion and Illness Prevention II
2.0K
The person's health status fluctuates continually, varying from being in good health to becoming ill and returning to being healthy. To understand the concept of illness prevention, there are two models. First, the health-illness continuum model is a graphic representation of an individual's wellness. It states that a person is considered healthy in the absence of physical disease and the presence of good emotional health.
The agent-host-environment model states that disease results...
The agent-host-environment model states that disease results...
2.0K
Regression Toward the Mean
6.8K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.8K
Mechanistic Models: Compartment Models in Individual and Population Analysis
230
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
230
Issues And Trends In Healthcare Delivery System
6.1K
The issues and trends in healthcare delivery are constantly changing. The COVID-19 pandemic is one recent issue that wreaked havoc on healthcare systems, causing a shortage of healthcare workers, high demand for medicines and supplies, and increased medical expenditure due to a lack of insurance. Other issues include rising healthcare costs and care fragmentation.
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
6.1K
Behavioral Genetics and Its Designs
978
Behavior genetics explores how genetic inheritance influences human behavior. It focuses on how genes, passed from parents to offspring, contribute to the development of behavioral traits and tendencies. This branch of genetics seeks to understand the complex interplay between inherited genetic factors and environmental influences in shaping our behaviors.
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
978

