Relational Graph Convolutional Network with BERT Embeddings for Ontology Relationship Classification

SNOMED CT is one of the largest and most widely used medical ontologies. It is a collection of medical concepts with a large number of relationships between concept pairs. One of the most laborious tasks for SNOMED CT curators is placing concepts in the correct hierarchical positions with accurate relationships. We present a method that utilizes domain-specific BERT models integrated with a Relational Graph Convolutional Network (RGCN) for classifying the relationships between SNOMED CT concept pairs. We generate node embeddings using five domain-specific BERT models, BioBERT, ClinicalBERT, SapBERT, SciBERT, and BioMedBERT, and feed them individually into an RGCN model for relationship classification. We apply a three-layer RGCN to capture and utilize the graph-structured dependencies among concepts, enabling the model to propagate relational information across connected nodes before performing classification. We demonstrate that each BERT embedding integrated with RGCN outperforms both the Message-Passing Neural Network (MPNN) and Neural Network (NN) baselines. Our work shows that SapBERT-RGCN achieves the best performance across all tested encoders, with an accuracy of 0.9394 and an F1-weighted score of 0.9412. Compared to the best-performing domain-specific BERT-NN baseline, this is a 4% accuracy gain with a notable improvement of F1-macro.