Development and Validation of a Natural Language Processing Algorithm to Pseudonymize Documents in the Context of a...

Area of Science:

• Medical Informatics
• Natural Language Processing
• Data Privacy

Background:

• Deidentifying clinical reports is crucial for research access.
• Patient privacy must be maintained during data sharing.
• Sharing deidentification tools and resources presents challenges.

Purpose of the Study:

• To implement systematic pseudonymization of clinical documents.
• To address challenges in deidentification tool sharing.
• To enable research access to clinical data while ensuring privacy.

Main Methods:

• Annotated a corpus of clinical documents with 12 entity types.
• Developed a hybrid deidentification system.
• Merged deep learning models with manual rules.

Main Results:

• Achieved an overall F1-score of 0.99 for deidentification.
• Analyzed factors influencing deidentification effort (dataset size, document types, models, rules).

Conclusions:

• The hybrid system effectively deidentifies clinical reports.
• Guidelines and code are shared to promote further research.
• Systematic pseudonymization enhances clinical data accessibility for research.