ドイツの臨床文献で否定検出のためのLLMの限界を探る
Richard Zowalla1,2, Martin Wiesner1
1Faculty for Informatics, Heilbronn University, Heilbronn, Germany.
Studies in health technology and informatics
|September 3, 2025
まとめ
大型言語モデル (LLM) は,ドイツ語の臨床テキストの否定を検出するのに有望ですが,ルールベースのシステムはより効率的です. 規則に基づく速度とLLMの精度をバランスにするハイブリッドアプローチは,実用的な臨床アプリケーションのために推奨されます.
科学分野:
- 医療情報学
- 自然言語処理
背景:
- 信頼性の高い文書化と情報に基づいた意思決定のために,臨床テキストの正確な否定検出は不可欠です.
- ドイツの臨床解約書簡は,否定の自動識別にユニークな課題を提示しています.
研究 の 目的:
- オープンソースのLarge Language Models (LLMs) が,ドイツの臨床解約書簡内の否定を検出する際のパフォーマンスを評価する.
- 確立されたルールベースのシステム (GeNeg) と人間の専門家アノテーションとの間でLLMのパフォーマンスを比較する.
主な方法:
- Llama 3.3 と Deepseek-R1 (70B) を含む複数のオープンソースのLLMを否定検出の正確性のために評価した.
- ルールベースのGeNegシステムと人間の注釈された基底の真実と比較したLLM結果.
- 各メソッドの計算時間とリソースの利用量を測定します.
主要な成果:
- Llama 3.3は最高精度 (.9670) とF1スコア (.9620) を示し,GeNegをわずかに上回った.
- 高精度にもかかわらず,LLMはGeNegと比較して大幅に高い計算コストと処理時間を誇示しました.
- ディープシーク-R1 (70B) も少量の精度向上を示したが,計算上の要求のために実用的ではなかった.
結論:
- LLMは臨床否定の検出に高い精度を提供していますが,その計算費用は即時の実用的な展開を制限しています.
- 将来の研究は,ルールベースの効率性とLLMの言語的強みを統合するハイブリッドモデルに焦点を当てるべきです.
- プロンプトを最適化し,LLMを伝統的な方法と組み合わせることで,精度と計算効率のバランスをとることができます.
さらに関連する動画
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
関連する概念動画
Difference from Background: Limit of Detection
7.1K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
7.1K
Negative and Cognitive Symptoms of Schizophrenia
120
Negative symptoms of schizophrenia indicate a reduction or absence of typical behaviors and emotional responses found in healthy individuals, while positive symptoms reflect an excess or distortion of normal functioning.
Negative Symptoms
Negative symptoms of schizophrenia manifest as deficits in normal emotional and behavioral functioning, profoundly impacting daily life. Individuals with schizophrenia often display a flat affect, characterized by a near-total absence of emotional expression,...
Negative Symptoms
Negative symptoms of schizophrenia manifest as deficits in normal emotional and behavioral functioning, profoundly impacting daily life. Individuals with schizophrenia often display a flat affect, characterized by a near-total absence of emotional expression,...
120
Negative Regulator Molecules
35.9K
Positive regulators allow a cell to advance through cell cycle checkpoints. Negative regulators have an equally important role as they terminate a cell’s progression through the cell cycle—or pause it—until the cell meets specific criteria.
35.9K
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
Detection of Black Holes
2.3K
Although black holes were theoretically postulated in the 1920s, they remained outside the domain of observational astronomy until the 1970s.
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
2.3K
Language and Cognition
438
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
438
