Comparing NER Approaches on French Clinical Text, with Easy-to-Reuse Pipelines

Area of Science:

• Natural Language Processing (NLP)
• Medical Informatics
• Computational Linguistics

Background:

• Clinical texts within Electronic Health Records (EHRs) are rich in information for observational studies.
• Heterogeneity of clinical texts across institutions and languages poses challenges for Named Entity Recognition (NER) tool performance.
• Standardized benchmarking is crucial due to the difficulty of sharing sensitive clinical data.

Purpose of the Study:

• To benchmark four Named Entity Recognition (NER) approaches on diverse French clinical corpora.
• To provide an open-source, reusable pipeline for evaluating NLP tools in the medical domain.
• To assess the impact of fine-tuning on existing NER models using clinical data.

Main Methods:

• Comparison of four NER approaches: three language model-based and one dictionary-based.
• Utilized three distinct French clinical corpora for evaluation.
• Developed and shared an open, reusable benchmarking pipeline using the medkit Python library.
• Included fine-tuning strategies with single or multiple corpora.

Main Results:

• Language models demonstrated superior performance compared to the dictionary-based approach.
• Fine-tuning on specific corpora did not consistently yield significant improvements over general biomedical models.
• The study highlights the effectiveness of modern NLP models for clinical text analysis.

Conclusions:

• Language models are highly effective for NER in French clinical texts.
• The necessity of extensive fine-tuning for pre-trained biomedical NLP models on specific clinical corpora is questioned.
• Sharing open and adaptable benchmarking pipelines is vital for advancing clinical NLP research.