Impact of De-Identification on Clinical Text Classification Using Traditional and Deep Learning Classifiers

Area of Science:

• Clinical informatics
• Natural Language Processing
• Machine Learning

Background:

• Clinical text de-identification is crucial for patient privacy and collaborative research.
• Concerns exist regarding the impact of de-identification on the utility of clinical text for downstream tasks like information extraction and machine learning.
• Evaluating the performance of machine learning models on de-identified clinical data is essential.

Purpose of the Study:

• To assess the impact of automatic clinical text de-identification on the performance of machine learning models for detecting altered mental status.
• To compare traditional machine learning models with deep learning models using both original and de-identified clinical notes.

Main Methods:

• Utilized a dataset of 1,113 emergency department history of present illness notes.
• Applied automatic de-identification to replace 1,795 protected health information tokens.
• Trained and evaluated traditional bag-of-words models and word-embedding based deep learning models.

Main Results:

• Deep learning models achieved the highest performance, with 95% accuracy on both original and de-identified notes.
• No statistically significant difference was observed in the performance of any tested models when comparing original versus de-identified notes.
• The de-identification process did not hinder the models' ability to detect altered mental status.

Conclusions:

• Automatic de-identification of clinical notes does not significantly impair the performance of machine learning models, including deep learning approaches.
• Clinical text de-identification is a viable strategy for enabling research while preserving patient confidentiality.
• Future research can confidently utilize de-identified clinical data for developing and deploying machine learning applications in healthcare.