Evaluating shallow and deep learning strategies for the 2018 n2c2 shared task on clinical text classification

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Clinical Informatics

Background:

• Automated clinical phenotyping is complex due to high-dimensional data and limited training sets, risking overfitting.
• Pretrained embeddings offer a potential solution by leveraging large-scale datasets for input representation.

Purpose of the Study:

• To evaluate shallow and deep learning text classifiers for automated clinical phenotyping.
• To assess the impact of pretrained embeddings on classifier performance with small clinical datasets.

Main Methods:

• Utilized a 2018 National NLP Clinical Challenges (n2c2) dataset of 202 patients for multilabel binary text classification.
• Compared a rule-based classifier, support vector machines, logistic regression, and long short-term memory networks.
• Evaluated logistic regression and long short-term memory with self-trained and pretrained BioWordVec embeddings.

Main Results:

• The rule-based classifier achieved the highest micro F1 score (0.9100), winning the challenge.
• Shallow machine learning methods outperformed deep learning and baseline classifiers.
• No significant difference in classification efficiency was observed between self-trained and pretrained embeddings.

Conclusions:

• Shallow methods, particularly rule-based approaches, demonstrate effectiveness in clinical phenotyping with small, imbalanced datasets.
• Deep learning struggled to capture term diversity in small datasets, while shallow methods were hindered by clinical complexities like negation.