Harnessing Moderate-Sized Language Models for Reliable Patient Data Deidentification in Emergency Department Records:...

Area of Science:

• Natural Language Processing
• Health Informatics
• Data Privacy

Background:

• Digitization of healthcare via electronic health records (EHRs) enhances research but raises privacy concerns.
• Machine learning and large language models (LLMs) have advanced patient data deidentification.
• Advanced LLMs face deployment challenges in hospitals due to security and hardware requirements.

Purpose of the Study:

• To design, implement, and evaluate deidentification algorithms using fine-tuned, moderate-sized open-source language models.
• To ensure the suitability of these models for production inference tasks on personal computers.
• To balance privacy preservation with textual integrity in clinical notes.

Main Methods:

• Utilized a dataset of over 425,000 clinical notes from Bordeaux University Hospital.
• Independently double-annotated 3000 notes for validation.
• Fine-tuned open-source models (Llama 2 7B, Mistral 7B, Mixtral 8x7B) using quantized low-rank adaptation.
• Evaluated PII-level (F1-score) and note-level (recall, BLEU) metrics.

Main Results:

• Mistral 7B achieved the highest overall F1-score (0.9673) and note-level recall (0.9326).
• Mistral 7B's recall for name deidentification reached 0.9915.
• BLEU scores consistently exceeded 0.9864, indicating minimal text alteration.

Conclusions:

• Generative NLP models, particularly Mistral 7B, demonstrate strong capabilities for efficient clinical text deidentification.
• Mistral 7B performs effectively on personal computers, addressing hardware limitations.
• This research facilitates broader access to pseudonymized clinical data for research and healthcare optimization.