提示是所有你需要的 - 直到它不是:在德国临床文本中探索否定检测LLM的极限
Richard Zowalla1,2, Martin Wiesner1
1Faculty for Informatics, Heilbronn University, Heilbronn, Germany.
Studies in health technology and informatics
|September 3, 2025
概括
大型语言模型 (LLM) 在检测德语临床文本中的否定方面表现有前途,但基于规则的系统仍然更有效. 建议在实践临床应用中使用混合方法,将LLM准确性与基于规则的速度进行平衡.
科学领域:
- 医疗信息学
- 自然语言处理
背景情况:
- 在临床文本中准确检测否定对于可靠的文档和明智的决策至关重要.
- 德国的临床出院信对自动化的否定识别提出了独特的挑战.
研究的目的:
- 评估开源大型语言模型 (LLM) 在德国临床出院信中检测否定的性能.
- 将LLM的表现与已建立的基于规则的系统 (GeNeg) 和人类专家的注释进行比较.
主要方法:
- 评估了多个开源LLM,包括Llama 3.3和Deepseek-R1 (70B),以检测否定的准确性.
- 将LLM结果与基于规则的GeNeg系统和人类注释的地面真相进行比较.
- 测量每个方法的计算时间和资源利用率.
主要成果:
- 拉玛 3.3显示出最高的精度 (.9670) 和F1得分 (.9620),略高于GeNeg.
- 尽管精度很高,但与GeNeg相比,LLM的计算成本和处理时间明显更高.
- 虽然Deepseek-R1 (70B) 的精度也略有提高,但由于计算需求而无法实现.
结论:
- 虽然LLM在临床否定检测中提供了高精度,但其计算成本限制了即时的实际部署.
- 未来的研究应侧重于将基于规则的效率与LLM语言优势相结合的混合模型.
- 优化提示并将LLM与传统方法相结合,可以在准确性和计算效率之间实现平衡.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
相关概念视频
Difference from Background: Limit of Detection
7.1K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
7.1K
Negative and Cognitive Symptoms of Schizophrenia
120
Negative symptoms of schizophrenia indicate a reduction or absence of typical behaviors and emotional responses found in healthy individuals, while positive symptoms reflect an excess or distortion of normal functioning.
Negative Symptoms
Negative symptoms of schizophrenia manifest as deficits in normal emotional and behavioral functioning, profoundly impacting daily life. Individuals with schizophrenia often display a flat affect, characterized by a near-total absence of emotional expression,...
Negative Symptoms
Negative symptoms of schizophrenia manifest as deficits in normal emotional and behavioral functioning, profoundly impacting daily life. Individuals with schizophrenia often display a flat affect, characterized by a near-total absence of emotional expression,...
120
Negative Regulator Molecules
35.9K
Positive regulators allow a cell to advance through cell cycle checkpoints. Negative regulators have an equally important role as they terminate a cell’s progression through the cell cycle—or pause it—until the cell meets specific criteria.
35.9K
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
Detection of Black Holes
2.3K
Although black holes were theoretically postulated in the 1920s, they remained outside the domain of observational astronomy until the 1970s.
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
2.3K
Language and Cognition
438
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
438
