Character-Level Neural Language Modelling in the Clinical Domain

Area of Science:

• Computational Linguistics
• Medical Informatics
• Artificial Intelligence

Background:

• Token-level word embeddings are standard for clinical natural language processing (NLP).
• Character-level neural language models (e.g., recurrent neural networks) are gaining attention for their comparable performance on NLP benchmarks.
• The applicability and semantic capture capabilities of character-level models in the clinical domain remain underexplored.

Purpose of the Study:

• To investigate the effectiveness of character-based language models in the clinical domain.
• To determine if character-level representations can capture meaningful lexical semantics in clinical text.
• To evaluate the necessity of token-based schemes for clinical NLP tasks.

Main Methods:

• Trained a long short-term memory (LSTM) network on German clinical problem list entries (50 characters each), linked to ICD-10 codes.
• Modeled the task as a time series of one-hot encoded single character inputs.
• Performed nearest neighbor search on character-induced word embeddings to find similar clinical concepts and evaluated captured semantics.

Main Results:

• Character-based models successfully captured traceable semantics at a syntactic level above individual characters.
• The models addressed the unique characteristics of clinical language.
• Evidence suggests character-level models can learn meaningful representations from clinical text.

Conclusions:

• Character-level language models are viable for clinical NLP, capturing relevant lexical semantics.
• These findings challenge the universal necessity of token-based representations in clinical NLP.
• Character-level models offer a fine-grained approach suitable for the complexities of clinical language.