大型语言模型与人类对临床文档的分类
Akram Mustafa1, Usman Naseem2, Mostafa Rahimi Azghadi1
1College of Science and Engineering, James Cook University, Townsville, 4811, QLD, Australia.
International journal of medical informatics
|January 23, 2025
概括
像ChatGPT 4这样的大型语言模型在改善国际疾病分类 (ICD-10) 编码准确性方面表现有前途,用于挑战医疗记录. 虽然人类编码器仍然优越,但ChatGPT 4实现了可比的中位数性能,这表明了增强临床文档的潜力.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 改善临床文档 改善临床文档
背景情况:
- 准确的国际疾病分类 (ICD-10) 编码对于临床文档至关重要.
- 机器学习和系统化医学命名法 (SNOMED) 地图显示出潜力,但与虚假负面作斗争.
- 在正确识别所有诊断方面仍然存在挑战,特别是在复杂的病例中.
研究的目的:
- 探索先进的大型语言模型 (LLM) 在提高ICD-10分类准确性的有效性.
- 解决当前机器学习和SNOMED映射在具有挑战性的医疗记录中的局限性.
- 评估LLM在之前被确定为假负的病例上的表现.
主要方法:
- 评估了ChatGPT 3.5和ChatGPT 4在ICD-10代码分类上的表现.
- 从密集护理医疗信息中心 (MIMIC) IV数据集中使用了802个具有挑战性的出院摘要 (先前方法的错误阴性).
- 将LLM成果与五位经验丰富的人类编码人员对100个摘要的子集的评估进行了比较.
主要成果:
- 与ChatGPT 3.5 (57-67%) 相比,ChatGPT 4的一致性显著更高 (86-89%).
- 总体而言,人类编码器的表现优于ChatGPT,但ChatGPT 4与人类编码器的中位数精度相匹配,为22%.
- 在不同的ICD-10代码中,ChatGPT 4的分类准确性有所不同.
结论:
- 先进的语言模型,特别是ChatGPT 4,显示了提高临床编码准确性的潜力.
- 将LLM与SNOMED映射等现有方法集成,可能会改善复杂案件的文档.
- 在具有挑战性的ICD-10分类场景中,ChatGPT 4为人类编码者提供了更好的一致性和可比的中位数性能.
相关概念视频
Methods of Documentation VI: Case Management Model
559
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
559
Classification of Illness
7.3K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.3K
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Clinical Trials: Overview
2.7K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.7K


