对15个自然语言处理算法的强大比较评估,以从二级护理记录中积极识别患有炎症性肠病的患者
Matt Stammers1,2, Markus Gwiggner3,4, Reza Nouraei2,5
1Department of Gastroenterology and The Southampton Emerging Therapies and Technologies (SETT) Centre, University Hospital Southampton NHS Foundation Trust, Southampton, UK m.stammers@soton.ac.uk.
评估开源自然语言处理 (NLP) 模型用于识别炎症性肠病 (IBD) 患者,发现虽然像DistilBERT这样的变压器模型表现出高性能,但大型语言模型 (LLM) 尽管目前存在成本和偏差问题,但仍有未来的希望.
科学领域:
- 计算语言学计算语言学
- 医疗信息学医学信息学
- 医疗保健中的人工智能
背景情况:
- 自然语言处理 (NLP) 有助于从临床笔记中识别炎症性肠病 (IBD) 的患者队列.
- 对于IBD队列识别的NLP的进展受到有限的代码,模型和数据集共享的阻碍.
研究的目的:
- 评估各种开源NLP模型,用于识别IBD患者队列.
- 评估NLP模型的文件到患者级分类性能,可解释性,可概括性,公平性和成本.
主要方法:
- 评估了15个NLP算法,涵盖基于规则,基于矢量和变压器架构,包括大型语言模型 (LLM).
- 评估模型使用混矩阵,计算时间,环境成本,公平度量和可解释性评估.
- 包括像spaCy,TF-IDF,sBERT,BERT变体 (DistilBERT,BioclinicalBERT,RoBERTa) 和多个LLMs这样的模型.
主要成果:
- 微调的DistilBERT_IBD模型获得了最高的微F1得分 (93.54%),紧随其后的是sBERT-Base (93.05%).
- 大型语言模型 (LLM) 在没有对特定数据集进行预先培训的情况下表现出强的表现 (微型F1: 86.47-92.20%),但却花费了大量的时间和成本.
- 对于表现最好的模型来说,特异性是一个显著的挑战,而在有效的模型类型中,偏差是一个重大的问题.
结论:
- NLP已经取得了显著的进步,LLM显示出从临床文本重新识别IBD患者的挑战的潜力.
- 预计在LLMs的未来进步将解决目前成本,性能和偏差方面的局限性.
- 一旦这些问题得到解决,LLM及其后继者将成为临床数据仓储中数据检索的主要工具.
更多相关视频
06:22Machine Learning-Based Cough Tone Classification: Diagnostic Exploration of Chronic Obstructive Pulmonary Disease and Respiratory Tract Infections
Published on: September 19, 2025
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
相关概念视频
Inflammatory Bowel Disease III: Diagnostic Studies and Management I-Nutritional Therapy
Diagnostic studies
A colonoscopy is the definitive screening test, distinguishing ulcerative colitis from other colon diseases with similar symptoms. During a colonoscopy test, inflamed mucosa with exudate ulcerations can be observed, and biopsies are taken to determine the histologic characteristics of the...
Inflammatory Bowel Disease II: Crohn's Disease
Inflammatory bowel disease, commonly known as IBD, refers to a collection of disorders that lead to persistent inflammation of the gastrointestinal tract. The two types of IBD are ulcerative colitis, which impacts the colon, and Crohn's disease, which can involve any part of the gastrointestinal segment.
Crohn's disease
Crohn's disease is a chronic, systemic inflammatory bowel disease (IBD) that predominantly affects the gastrointestinal tract. It is marked by...
Inflammatory Bowel Disease V: Surgical Management
Here are some common surgical interventions for IBD:
Chronic Bowel Disorders: Introduction
Irritable Bowel Syndrome (IBS) is a common disorder affecting the gastrointestinal tract. The distinctive feature is recurrent abdominal pain associated with altered bowel movements, manifesting as constipation, diarrhea, or fluctuating between both. The...
Irritable Bowel Syndrome II: Clinical Features and Diagnostic Evaluation
Irritable Bowel Syndrome (IBS) is classified into subtypes based on the predominant bowel habits as determined by the Bristol Stool Form Scale (BSFS). The subtypes are:
Inflammatory Bowel Disease I: Ulcerative Colitis
Inflammatory bowel disease, or IBD, encompasses a group of disorders characterized by chronic inflammation or ulceration of the gastrointestinal tract.
Risk Factors
The exact cause of IBD remains unclear, although it is believed to be due to a mix of genetic, environmental, microbial, and immune factors. Genetic factors are significant in determining susceptibility to IBD, with family history being a critical risk factor. Individuals with a first-degree relative who has IBD are at...
