The Limits of Generalization: Zero-Shot French Medical NER Using French, English and Multilingual GLiNER Models

This study evaluates zero-shot Named Entity Recognition (NER) using several GLiNER-based models on French medical text. Eight open datasets covering diseases, symptoms, and drugs are used to assess generalization across varied formats and domains. Models are evaluated using entity-level F1-scores based on MUC-5 metrics, with prompts formulated using English, French and bilingual labels to assess cross-lingual robustness. Results show that OpenMed models, explicitly trained for the involved medical entities, outperform general and domain-specialized GLiNER variants. However, performance varies by dataset and context, revealing challenges in transfer learning and generalizability. The findings underscore the importance of developing zero-shot NER models resilient to dataset biases and contexts.