Towards a ModernBERT Model Adapted to the Biomedical Domain in Italian

We present bio-modernbert-ita, a domain-adapted Italian Modern-BERT model specialized for biomedical text. Given the scarcity of biomedical Italian data, we translated 22 million PubMed abstracts to obtain our training corpus. We then continued pre-training the model and fine-tuned it for named entity recognition. We evaluated it with two different datasets to assess its performance against the original (baseline) model. Our model reaches an F1-score of 56.7% on E3C and 75.9% on PharmaER.IT with respective improvements of +4.9% and +6.5% over the baseline model. These results suggest that bio-modernbert-ita can provide a starting point for Italian biomedical models. Future work will focus on completing pre-training, training for the clinical domain, and broadening the evaluation set.