对于临床NLP的DRAGON基准标准
Joeran S Bosma1,2,3, Koen Dercksen4, Luc Builtjes4
1Diagnostic Image Analysis Group, Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands. Joeran.Bosma@radboudumc.nl.
NPJ digital medicine
|May 16, 2025
概括
DRAGON挑战基准提高了使用大型语言模型 (LLM) 的临床自然语言处理 (NLP). 特定领域的预训练显著提高了医疗数据注释任务的性能.
科学领域:
- 医疗信息学 医疗信息学
- 计算语言学 计算语言学
- 人工智能的人工智能
背景情况:
- 人工智能 (AI) 可以解决医疗诊断人员短缺的问题.
- 大规模的注释数据集对于训练临床AI算法至关重要.
- 自然语言处理 (NLP),特别是大型语言模型 (LLM),对临床数据注释有希望,但缺乏公共基准.
研究的目的:
- 介绍DRAGON挑战,这是临床NLP的一个基准.
- 促进临床数据的自动化,大规模和具有成本效益的注释.
- 评估不同预训练策略对临床NLP任务的LLM绩效的影响.
主要方法:
- 开发了DRAGON挑战,其中包括28个任务和28824份来自五个荷兰护理中心的注释医疗报告.
- 在荷兰第六个护理中心的400万份临床报告上培训了基础的LLM.
- 使用特定领域,混合领域和一般领域预培训策略评估LLM绩效.
主要成果:
- 特定领域的预训练取得了最高的成绩 (DRAGON 2025测试得分为0.770),超过混合领域 (0.756) 和一般领域的预训练 (0.734,p < 0.005).
- 在28个任务中的18个任务中观察到强的表现.
- 在10项任务上Subpar的表现突出显示了临床NLP需要进一步创新的领域.
结论:
- 在临床NLP任务中,特定领域的预训练优越.
- DRAGON挑战基准为推进临床NLP提供了宝贵的资源.
- 公共可用的基准,代码和基础的LLM将加速自动化临床数据注释的研究和开发.
更多相关视频
相关概念视频
01:16Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
01:10Higher Mental Functions of the Brain: Language
700
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
700
01:11Clinical Trials: Overview
2.6K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.6K
01:27Nursing Clinical Information System
733
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
733
02:34Empathy
9.5K
Some researchers suggest that altruism operates on empathy. Empathy is the capacity to understand another person’s perspective, to feel what he or she feels. An empathetic person makes an emotional connection with others and feels compelled to help (Batson, 1991). Empathy can be expressed in several ways, including cognitive, affective, and motor.
9.5K
01:25Neural Circuits
952
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
952