miCDER: a context-aware transformer model for joint miRNA-disease entity and multi-level regulatory relation...

Dysregulation of microRNAs (miRNAs) is closely linked to the progression of diverse human diseases. However, the lack of a standardized, fine-grained dataset of miRNA-disease regulatory interactions and the limited ability of existing methods to capture multi-level regulatory relations hinder full understanding of these mechanisms.

We constructed a fine-grained, multi-level annotated biomedical dataset covering ten entity types and thirteen relation categories, tailored for modeling microRNA-disease interactions. We then propose miCDER, a transformer-based model that jointly extracts entities and relations through contextual encoding and inter-span transformer attention. miCDER achieves state-of-the-art performance in both named entity recognition (NER F1 = 87.34%) and relation extraction (RE F1 = 77.28%), significantly outperforming all methods, including SpERT, STER, BiLSTM-CRF, and CNN. Compared to the strongest baseline SpERT, our model achieved improvements of 8.84% in NER F1 and 12.35% in RE F1, respectively. To validate generalizability, we evaluated miCDER on two public benchmark datasets and compared it with state-of-the-art methods, achieving competitive performance on CoNLL04 (NER F1 = 88.83%, RE F1 = 72.31%) and superior results on the biomedical ADE dataset (NER F1 = 90.42%, RE F1 = 82.81%). Applied to 27,051 curated miRNA-related text segments from PubMed, miCDER automatically extracted 93,221 high-confidence regulatory triplets to construct the miRNA-disease knowledge graph, MAAD-HCD-KG. Among these, 1,735 targeting relations not found in the microRNA target database (miRTarBase) demonstrate miCDER's ability to capture literature-supported regulatory associations that are not yet included in manually curated databases.

Our proposed miCDER model can serve as an effective tool for extracting fine-grained regulatory information from biomedical texts. The resultant knowledge graph constitutes a significant resource for the advancement of research on diseases related to microRNA.