An Iterative Composite Method for the De-Identification of Medical Notes

Area of Science:

• Medical Informatics
• Natural Language Processing
• Health Data Security

Background:

• The advancement of artificial intelligence (AI) and large language models (LLMs) in healthcare necessitates access to deidentified patient data.
• Electronic medical records (EMRs) contain sensitive patient information requiring robust de-identification methods.
• Current de-identification techniques may struggle with residual identifiers, posing privacy risks.

Purpose of the Study:

• To develop and evaluate an iterative de-identification method for medical notes.
• To enhance the security and privacy of electronic medical record (EMR) data for AI applications.
• To address limitations in existing de-identification approaches by incorporating multiple techniques.

Main Methods:

• An iterative de-identification process combining pattern matching, dictionary-based identification, and contextual token analysis.
• Implementation of an iterative review step to identify and remove residual identifiers.
• Application of the method to a large dataset of 350,000 deidentified medical notes.

Main Results:

• The proposed iterative de-identification method demonstrated strong performance.
• Preliminary evaluations achieved a high F1 score of 0.992.
• The method proved effective in de-identifying a diverse dataset of medical notes.

Conclusions:

• The developed iterative de-identification method is effective and robust for medical notes.
• This approach facilitates the safe and ethical use of EMR data for AI and LLM development in healthcare.
• The study highlights the importance of multi-faceted strategies for comprehensive medical data de-identification.