CaseReportBench:在临床病例报告中用于密集信息提取的LLM基准数据集
Xiao Yu Cindy Zhang1, Carlos R Ferreira2, Francis Rossignol2
1University of British Columbia.
概括
大型语言模型 (LLM) 可以从罕见疾病病例报告中提取重要的临床细节,以帮助诊断. 新的提示策略提高了LLM在这个任务上的表现,推进了医疗AI应用.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 人工智能在医学中的应用
背景情况:
- 罕见的疾病,如新陈代谢的先天性错误 (IEM),存在重大诊断障碍.
- 病例报告是诊断罕见疾病的有价值但未得到充分利用的资源.
- 需要计算方法从病例报告中提取密集的临床信息.
研究的目的:
- 介绍CaseReportBench,这是一个用于从IEM案例报告中提取密集信息的新型数据集.
- 评估大型语言模型 (LLM),从病例报告中提取临床信息.
- 开发和评估先进的提示策略,以提高LLM绩效.
主要方法:
- 开发CaseReportBench,一个专家注释的数据集,用于密集的信息提取.
- 评估各种LLM和提示技术,包括特定类别的提示和分类过的数据集成.
- 临床医生对LLM提取的信息进行临床相关性的评估.
主要成果:
- 特定类别的提示显著改善了与基准数据集的LLM对齐.
- 与GPT-4o相比,开源Qwen2.5:7B在此任务中表现出优越的性能.
- LLMs成功地提取了临床相关的细节,支持罕见疾病的诊断和管理.
结论:
- 在从病例报告中提取关键信息以帮助诊断罕见疾病方面,LLM是有前途的.
- 新的提示策略增强了临床NLP任务的LLM能力.
- 需要进一步改进,特别是在识别差异诊断的负面发现方面.
相关概念视频
Clinical Trials: Overview
4.6K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
4.6K
Clinical Trials
10.1K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
10.1K
Nursing Clinical Information System
1.2K
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
1.2K
Classification of Illness
8.6K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.6K
Data Collection I
7.8K
Data collection gathers information needed to make accurate judgments about a patient's present condition. During a health history interview, subjective data is collected from the patient, their caregivers, or family members, and objective data is collected through observations and physical assessment. Patients are the primary source of subjective data. Thus information gathered from patients through interviews, observations, and physical examination is primary data. Secondary sources of...
7.8K


