The Limits of Generalization: Zero-Shot French Medical NER Using French, English and Multilingual GLiNER Models.

Area of Science:

• Natural Language Processing (NLP)
• Computational Linguistics
• Medical Informatics

Background:

• Zero-shot Named Entity Recognition (NER) enables models to identify entities without specific training data.
• Evaluating NER models on diverse French medical datasets is crucial for clinical applications.
• GLiNER-based models offer a promising framework for adaptable NER tasks.

Purpose of the Study:

• To evaluate the performance of GLiNER-based models for zero-shot NER on French medical text.
• To assess the generalization capabilities of these models across various medical datasets (diseases, symptoms, drugs).
• To investigate the impact of prompt language (English, French, bilingual) on cross-lingual robustness.

Main Methods:

• Utilized eight open French medical datasets covering diseases, symptoms, and drugs.
• Employed GLiNER-based models, including general, domain-specialized, and OpenMed variants.
• Evaluated models using entity-level F1-scores with MUC-5 metrics.
• Assessed cross-lingual robustness by formulating prompts in English, French, and bilingual formats.

Main Results:

• OpenMed models, trained on specific medical entities, demonstrated superior performance compared to general and other domain-specialized GLiNER models.
• Model performance exhibited significant variation across different datasets and contexts.
• Cross-lingual prompt formulation showed an impact on robustness, though specific results varied.

Conclusions:

• Specialized models (OpenMed) are more effective for French medical zero-shot NER than general or less specialized variants.
• Challenges remain in achieving consistent generalizability and transfer learning across diverse medical text contexts.
• Future research should focus on developing zero-shot NER models that are more resilient to dataset biases and contextual variations in the medical domain.