CONORM-DEID: Robustness Evaluation of a Multilingual De-Identification System for Clinical Texts

Clinical narratives contain rich, detailed information that is essential for medical research but often locked behind privacy constraints due to the presence of personally identifiable information. To enable secondary use while protecting patient confidentiality, these texts must undergo de-identification. Because even a few missed instances can lead to disclosure, systems must achieve high recall. In this work, we present CONORM-DEID, a multilingual model for clinical text de-identification, and investigate how large-scale synthetic pre-training and the availability of human-annotated data impact model robustness in both English and French. We compare models trained from scratch with models incorporating synthetic pre-training across high- and low-data settings. All models trained from scratch achieve high F1-scores (97%+), but robustness, measured as precision at 99% recall, improves only when synthetic pre-training is followed by substantial human fine-tuning. When annotated data are limited, pre-training instead reduces performance, indicating negative transfer. Our code is publicly available at https://github.com/ds4dh/multilingual_deidentification.