Fusing Semantic and Structural Features for Code Error Detection

Large Language Models of the Transformer architecture display great promise in automated code error detection based on their strength in processing sequential data. Nevertheless, their efficacy could be further improved by addressing the inherent weakness in handling structural code dependencies. In response to this, we introduce a novel model that integrates the semantic comprehension power of RoBERTa with the structural learning strength of Graph Neural Networks. This model aims to detect the most common categories of programming faults in the form of runtime errors, index errors, and import/module errors. Experimental evaluation has demonstrated that the hybrid model, utilizing a proper fusion technique, outperforms other models in terms of accuracy and robustness. The introduced mechanism leads to numerical benefits, improving test accuracy by 1.75% over competitive baseline.