De-identification of clinical notes in French: towards a protocol for reference corpus development

Area of Science:

• Natural Language Processing
• Medical Informatics
• Computational Linguistics

Background:

• Facilitating research in Natural Language Processing (NLP) for clinical documents requires robust tools for automatic de-identification of Electronic Health Records (EHRs).
• Developing high-quality reference corpora is essential for training and evaluating de-identification systems.

Purpose of the Study:

• Investigate methods for creating a high-quality reference corpus for de-identifying French clinical documents.
• Compare the effectiveness of different automatic de-identification systems in pre-annotating clinical text for human revision.

Main Methods:

• A diverse corpus of French clinical documents was pre-processed using a rule-based system and a Conditional Random Fields (CRF) system.
• Human annotators revised pre-annotated documents, marking ten categories of Protected Health Information (PHI), working independently and blind to system outputs.
• The best pre-annotation system was used to create a training set for a statistical de-identification model.

Main Results:

• The rule-based system achieved better pre-annotation performance (F=0.813) than the CRF system (F=0.519).
• Human revision time was longer for the rule-based system's pre-annotations, but resulted in higher quality annotations (F-measure up to 0.987).
• A statistical de-identification system trained on only 20 custom documents outperformed pre-annotation systems trained on external corpora.

Conclusions:

• High-quality pre-annotations improve reference corpus quality but increase revision time.
• Statistical de-identification methods demonstrate superior performance with limited custom training data compared to rule-based approaches.