Customization scenarios for de-identification of clinical notes

Area of Science:

• Medical Informatics
• Machine Learning
• Data Privacy

Background:

• Automated machine learning systems can de-identify electronic medical records (EMRs), including clinical notes.
• Wider use of these systems is hindered by performance uncertainties on new datasets.

Purpose of the Study:

• To assess the performance of various machine learning (ML) de-identification systems, from off-the-shelf to fully customized.
• To provide practical options for clinical note de-identification.

Main Methods:

• Implemented a state-of-the-art ML de-identification system.
• Trained and tested systems on matched dataset pairs simulating deployment scenarios.
• Utilized clinical notes from i2b2, Physionet Gold Standard, and MIMIC-III datasets.

Main Results:

• Fully customized systems achieved 97-99% removal of personally identifying information.
• Off-the-shelf system performance varied by dataset, generally exceeding 90%.
• Fine-tuning with small labeled or large unlabeled datasets enhanced performance over standard off-the-shelf models.

Conclusions:

• Health organizations must consider available customization levels when selecting de-identification solutions.
• Matching deployment solutions to organizational resources and performance targets is essential for effective EMR de-identification.