从临床文档中提取疾病国际分类代码,使用大型语言模型
Ashley Simmons1, Kullaya Takkavatakarn2,3, Megan McDougal1
1Department of Human Performance - Health Informatics and Information Management, West Virginia University, Morgantown, West Virginia, United States.
Applied clinical informatics
|November 28, 2024
概括
大型语言模型 (LLM) 在从患者笔记中提取国际疾病分类,第十次修订,临床修改 (ICD-10-CM) 代码时显示出有限的准确性,与人类编码器相比,协议最小.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床文档 临床文档
背景情况:
- 大型语言模型 (LLM) 在医学和法律等专业领域展示了潜力.
- 它们在专业医疗任务中的有效性,如ICD-10-CM代码提取,尚未得到充分证实.
研究的目的:
- 评估和比较各种LLM在提取ICD-10-CM代码与人类编码器中的性能.
- 确定基于LLM的代码提取中的特定挑战和差异原因.
主要方法:
- 六个LLM (GPT-3.5,GPT-4,克劳德2.1,克劳德3,双子座高级,拉玛2-70b) 进行了评估.
- 来自AHIMA Vlab的非识别住院病人的笔记被用于评估.
- 计算了百分比协议和科恩的卡帕;在子集中分析了差异.
主要成果:
- 与人类编码者相比,LLM提取了更多独特的ICD-10-CM代码,但一致性很低 (kappa值: -0.02至0.01).
- 对于所有代码,GPT-4显示了最高的百分比一致性 (15.2%);克劳德3在初级诊断方面领先 (26%一致性,0.25卡帕).
- 差异包括未经证实的诊断,非特定的代码,症状编码和幻觉.
结论:
- 目前的LLM在从住院病人的笔记中提取ICD-10-CM代码方面表现不佳.
- 对于医学编码中LLM可靠的临床应用,需要进行显著的改进.
相关概念视频
Classification of Illness
7.4K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.4K
Nursing Interventions II: Selecting and Classifying the Nursing Interventions
2.1K
Creating and executing a nursing diagnosis helps nurses plan care and guide patient, family, and community interventions. They are developed based on a patient's physical evaluation and support measuring the outcomes. It is not recommended to select random interventions throughout the planning process. Instead, consider the following six essential factors when choosing interventions:
2.1K
Methods of Documentation VI: Case Management Model
559
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
559
Formulating and Validating Nursing Diagnosis I
2.6K
A nursing diagnosis is written when the nurse recognizes a cluster of essential patient data indicating health problems treated with independent nursing interventions. The standardized terminologies of a nursing diagnosis help nurses identify and treat patients' problems. Every electronic health record that uses nursing diagnosis must employ standard diagnostic terminology. Developing an efficient, individualized care plan begins with accurate nursing diagnoses.
There are thirteen domains...
There are thirteen domains...
2.6K
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Nursing Clinical Information System
754
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
754


