Speculation detection for Chinese clinical notes: Impacts of word segmentation and embedding models

Area of Science:

• Natural Language Processing (NLP)
• Clinical Informatics
• Computational Linguistics

Background:

• Identifying speculative statements in clinical texts is crucial for accurate information extraction.
• Chinese clinical notes present unique challenges for NLP due to the necessity of word segmentation.
• Existing methods may not fully leverage domain-specific linguistic features for speculation detection.

Purpose of the Study:

• To develop a state-of-the-art speculation detection system for Chinese clinical notes.
• To evaluate the impact of embedding features and word segmentation on system performance.
• To investigate the utility of domain-specific word segmentation for clinical NLP.

Main Methods:

• A sequence labeling approach was employed for speculation detection.
• Features included bag of characters, bag of words, character embeddings, and word embeddings.
• Experiments were conducted on a dataset of 36,828 clinical notes with 5103 annotations.
• Performance was compared using general and domain-specific word segmenters.

Main Results:

• The developed system achieved a high performance of 92.2% F-score.
• Word segmentation was demonstrated to be critical for high-quality word embeddings.
• A domain-dependent word segmenter significantly enhanced the performance of the speculation detection system.

Conclusions:

• The proposed system effectively detects speculations in Chinese clinical notes.
• High-quality word embeddings, derived from accurate word segmentation, are vital for clinical NLP.
• Domain-specific word segmentation is essential for optimizing NLP tasks in the Chinese clinical domain.