Using Character-Level and Entity-Level Representations to Enhance Bidirectional Encoder Representation From...

Area of Science:

• Natural Language Processing (NLP)
• Medical Informatics
• Computational Linguistics

Background:

• Electronic Health Records (EHRs) improve care but suffer from low-quality content due to copy-paste and template usage.
• The 2019 national NLP clinical challenge (n2c2) focused on Clinical Semantic Textual Similarity (ClinicalSTS) to address data redundancy.
• Accurate semantic similarity is crucial for managing and interpreting clinical data.

Purpose of the Study:

• To investigate novel methods for modeling ClinicalSTS.
• To analyze the performance of a semantically enhanced text matching model.

Main Methods:

• Developed a model with character-level (CNN), sentence-level (BERT), and entity-level representations.
• Compared two entity-level encoding methods: entity-type labels and MeSH knowledge graph representations.
• Evaluated the model on the ClinicalSTS corpus from the 2019 n2c2/OHNLP challenge.

Main Results:

• A BERT-only model achieved a Pearson correlation coefficient (PCC) of 0.848.
• Adding character-level and entity-level representations individually improved PCC to 0.857/0.854 and 0.859.
• Combining all representations yielded the highest PCC of 0.861 (entity I) and 0.868 (entity II).

Conclusions:

• Character-level and entity-level information significantly enhance BERT-based STS models.
• The proposed model demonstrates improved performance in ClinicalSTS tasks.
• Integrating diverse data representations is key to advancing clinical NLP.