Technical Note: An embedding-based medical note de-identification approach with sparse annotation.

Area of Science:

• Natural Language Processing
• Medical Informatics
• Machine Learning

Background:

• Medical note de-identification is crucial for protecting patient privacy and enabling secure data sharing in research.
• Current methods like pattern matching and entity tagging have limitations, including extensive annotation requirements and format variations.

Purpose of the Study:

• To develop a novel de-identification approach with a reduced annotation burden.
• To implicitly account for the language territory of sensitive information in medical records.

Main Methods:

• A sparsely supervised classification method using a contextualized word embedding module and a multilayer perceptron.
• The approach infers vocabulary similarity to a landmark set for sensitive and non-sensitive information.
• Demonstrated with name removal, the method is applicable to other sensitive strings.

Main Results:

• Achieved >0.99 accuracy on a large cohort of hybrid clinical reports.
• Demonstrated high sensitivity (1.0) and specificity (0.9973).
• Outperformed the Stanford NER tagger in F1 score (0.889 vs. 0.822).

Conclusions:

• The proposed method offers a more effective alternative to existing de-identification techniques.
• Achieved superior qualitative and quantitative results compared to the pretrained Stanford NER toolbox.