大型语言模型在生成合成电子健康记录中的评估和偏差分析:比较研究
Ruochen Huang1, Honghan Wu2, Yuhan Yuan1
1School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China.
Journal of medical Internet research
|May 12, 2025
概括
大型语言模型 (LLM) 生成合成电子健康记录 (EHR),完整性提高,但性别和种族偏见放大. 解决这些绩效偏见的权衡对于公平的医疗保健AI至关重要.
科学领域:
- 医疗保健中的人工智能
- 医疗信息学 医疗信息学
- 健康 公平 卫生 公平
背景情况:
- 由大型语言模型 (LLM) 生成的合成电子健康记录 (EHR) 为临床教育和模型培训提供保护隐私的解决方案.
- 然而,LLM产生的电子健康记录中的未经探索的绩效差异和人口偏见对公平的医疗保健构成风险.
研究的目的:
- 系统地评估各种LLM在生成合成EHR方面的表现.
- 批判性地评估LLM生成的EHR中的性别和种族偏见,涉及20种不同的人口流行病的疾病.
主要方法:
- 开发了一个框架,使用7个LLM和10个提示生成14万个合成EHR.
- 引入了电子医疗记录绩效评分 (EPS) 用于完整性和统计平价差异 (SPD) 用于人口偏差评估.
- 利用千平方测试来评估人口群体之间的偏见.
主要成果:
- 较大的LLM表现出优越的EHR生成性能 (更高的EPS),但表现出增加的性别和种族偏见.
- 观察到性别两极分化,女性占主导地位的疾病的代表性扩大,其他疾病的男性代表性偏差.
- 确定了种族偏见,包括高估白人/黑人人口和低估西班牙裔/亚洲裔群体在合成电子健康记录中.
结论:
- 存在业绩偏差的权衡:较大的LLM产生更全面的EHR,但具有较高的人口偏差.
- 在所有测试的LLM中都存在偏差,而不仅仅是较大的模型.
- 调查结果强调,迫切需要在医疗保健AI中制定偏见缓解策略和公平性基准.
相关概念视频
Methods of Documentation VII: EMR
782
Electronic Medical Records (EMRs) primarily center around electronically documenting patients' health information within a single healthcare organization or practice. They contain essential clinical data related to a patient's medical history, diagnoses, medications, treatment plans, lab results, and other pertinent information relevant to the specific encounter or episode of care. EMRs are designed to streamline documentation and workflow processes within individual healthcare...
782
Purpose of Health Records I
1.1K
The vital purpose of health records is to provide a complete and accurate account of a patient's medical history, including communication, diagnostic and therapeutic orders, care planning, research, and quality review.
Here's a breakdown of how health records serve these purposes:
Here's a breakdown of how health records serve these purposes:
1.1K
Issues And Trends In Healthcare Delivery System
5.5K
The issues and trends in healthcare delivery are constantly changing. The COVID-19 pandemic is one recent issue that wreaked havoc on healthcare systems, causing a shortage of healthcare workers, high demand for medicines and supplies, and increased medical expenditure due to a lack of insurance. Other issues include rising healthcare costs and care fragmentation.
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
5.5K
Bias in Epidemiological Studies
99
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
99
Mechanistic Models: Compartment Models in Individual and Population Analysis
19
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
19


