Does BERT need domain adaptation for clinical negation detection?

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Machine Learning

Background:

• Negation detection in clinical text is a critical but challenging task.
• Domain adaptation and transfer learning offer potential solutions for improving negation detection.
• Understanding the interplay between Bidirectional Encoder Representations from Transformers (BERT) and domain adaptation is crucial.

Purpose of the Study:

• To investigate neural unsupervised domain adaptation methods for clinical negation detection.
• To evaluate the effectiveness of combining domain adaptation with BERT for negation detection.
• To analyze the interaction between BERT and domain adaptation techniques.

Main Methods:

• Utilized four clinical text datasets annotated for negation status.
• Evaluated a neural unsupervised domain adaptation algorithm and BERT.
• Developed a BERT extension incorporating domain adversarial training.

Main Results:

• Domain adaptation methods showed positive results but did not outperform plain BERT.
• BERT demonstrated superior performance in clinical negation detection compared to domain adaptation.
• Evidence suggests BERT's gains are not additive with domain adaptation gains.

Conclusions:

• BERT subsumes domain adaptation for clinical negation detection, indicating its robust generalization capabilities.
• BERT's extensive pre-training enables effective learning of general negation representations.
• Fine-tuning BERT on specific corpora does not appear to lead to significant overfitting.