大型语言模型在提取IBD患者报告结果时优于传统的自然语言处理方法
Perseus V Patel1,2, Conner Davis3, Amariel Ralbovsky1
1Department of Pediatrics, University of California San Francisco, San Francisco, CA.
medRxiv : the preprint server for health sciences
|September 16, 2024
概括
大型语言模型 (LLM) 准确地从炎症性肠病 (IBD) 临床笔记中提取患者报告的结果 (PRO). 与传统的NLP方法相比,LLM在各个机构中显示出更高的概括性和准确性.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 胃肠病学 胃肠病学
背景情况:
- 患者报告的结果 (PROs) 对于评估炎症性肠病 (IBD) 活动和治疗有效性至关重要.
- 从临床笔记中手动提取PRO是耗时和劳动密集的.
- 改善电子健康记录 (EHR) 的数据策划对于研究和质量改进至关重要.
研究的目的:
- 将传统的自然语言处理 (tNLP) 与大型语言模型 (LLM) 进行比较,以提取IBD PROs.
- 评估LLMs在从临床笔记中提取腹痛,腹和便血液方面的准确性和概括性.
- 评估LLM在增强IBD研究和患者护理方面的潜力.
主要方法:
- 使用预定义的协议,对三个IBD PRO的临床注释进行注释.
- 在UCSF开发和内部测试tNLP和LLM模型.
- 在斯坦福大学对模型进行外部验证,比较准确度,灵敏度,特异性和预测值.
- 对所有模型进行了公平性和错误评估.
主要成果:
- 评价者之间的注释可靠性超过了90%.
- 在内部测试中,LLM (GPT-4) 达到96% (腹痛),88% (腹) 和90% (便血液) 的准确度,与顶级tNLP模型相美.
- 外部验证显示tNLP模型未能概括 (61-62%的准确性),而GPT-4保持了>90%的准确性;PaLM-2和GPT-4的表现类似. 没有检测到人口偏见.
结论:
- 大型语言模型为从IBD临床笔记中提取PRO提供了准确和可概括的方法.
- 在不同机构中,LLM保持高准确度,不管笔记模板和作者的变化如何.
- 广泛采用LLM可以显著推进IBD研究和改善患者护理.
相关概念视频
Inflammatory Bowel Disease III: Diagnostic Studies and Management I-Nutritional Therapy
324
Various diagnostic tests are employed in the diagnostic process for Inflammatory Bowel Disease (IBD), particularly to differentiate between Crohn's disease and ulcerative colitis.
Diagnostic studies
A colonoscopy is the definitive screening test, distinguishing ulcerative colitis from other colon diseases with similar symptoms. During a colonoscopy test, inflamed mucosa with exudate ulcerations can be observed, and biopsies are taken to determine the histologic characteristics of the...
Diagnostic studies
A colonoscopy is the definitive screening test, distinguishing ulcerative colitis from other colon diseases with similar symptoms. During a colonoscopy test, inflamed mucosa with exudate ulcerations can be observed, and biopsies are taken to determine the histologic characteristics of the...
324
Inflammatory Bowel Disease IV: Pharmacological Management
118
Upon diagnosis, managing Inflammatory Bowel Disease (IBD) involves addressing several crucial aspects. The primary goals include resting the bowel, correcting malnutrition, and providing symptomatic relief. Resting the bowel may consist of medications to reduce inflammation and promote healing. Correcting malnutrition is essential, often requiring dietary adjustments and nutritional supplements. Symptomatic relief aims to ease pain, diarrhea, and other discomforts in IBD.
Pharmacologic...
Pharmacologic...
118
Drugs for Treatment of Crohn's Disease in IBD Using Glucocorticoids
98
Glucocorticoids, a class of anti-inflammatory drugs, are pivotal in treating moderate to severe Crohn's disease by inducing remission. They exhibit their anti-inflammatory action by inhibiting the production of inflammatory cytokines such as tumor necrosis factor (TNF)-α, interleukin (IL)-1, and chemokines like IL-8. In addition, they reduce the expression of inflammatory cell adhesion molecules and inhibit gene transcription of nitric oxide synthase, phospholipase A2, cyclooxygenase-2...
98
Inflammatory Bowel Disease V: Surgical Management
128
Surgical interventions for inflammatory bowel disease (IBD), which includes ulcerative colitis and Crohn's disease, are essential in managing symptoms and addressing complications. The selection of surgical procedures is contingent upon the specific conditions and complications that stem from these illnesses.
Here are some common surgical interventions for IBD:
Here are some common surgical interventions for IBD:
128
Inflammatory Bowel Disease II: Crohn's Disease
201
Introduction
Inflammatory bowel disease, commonly known as IBD, refers to a collection of disorders that lead to persistent inflammation of the gastrointestinal tract. The two types of IBD are ulcerative colitis, which impacts the colon, and Crohn's disease, which can involve any part of the gastrointestinal segment.
Crohn's disease
Crohn's disease is a chronic, systemic inflammatory bowel disease (IBD) that predominantly affects the gastrointestinal tract. It is marked by...
Inflammatory bowel disease, commonly known as IBD, refers to a collection of disorders that lead to persistent inflammation of the gastrointestinal tract. The two types of IBD are ulcerative colitis, which impacts the colon, and Crohn's disease, which can involve any part of the gastrointestinal segment.
Crohn's disease
Crohn's disease is a chronic, systemic inflammatory bowel disease (IBD) that predominantly affects the gastrointestinal tract. It is marked by...
201
Drugs for Treatment of Crohn's Disease in IBD Using Biologic Agents: Anti-TNF
120
Tumor Necrosis Factor (TNF), a proinflammatory cytokine, contributes significantly to the inflammation seen in Crohn's disease. It exists as soluble TNF and membrane-bound TNF, with actions mediated through TNF receptors (TNFR). TNFR activation leads to the release of proinflammatory cytokines, T-cell activation, collagen production, and leukocyte migration, all contributing to inflammation in Crohn's disease. Anti-TNF monoclonal antibodies, namely infliximab (Remicade), adalimumab...
120


