GRAFF: GRaph-Augmented Fine-grained Fusion for Large Language Models

Recent advancements in large language models (LLMs) have showcased remarkable text generation capabilities. However, due to the inherent ambiguity of natural language and the unstructured nature of text modality, LLMs still struggle to integrate structured information (e.g., graphs) effectively. This hinders their ability to leverage high-quality structured data in specialized domains. Thus, recent research has explored various methods to integrate graph structures into LLMs to improve generation. However, existing methods typically compress the graph's structural information into only a single token, which is concatenated with detailed text tokens for LLMs, restricting their ability to capture deep semantic and structural information. To overcome these limitations, we propose GRaph-Augmented Fine-grained Fusion (GRAFF), a novel method that integrates fine-grained node-level structural information with corresponding text entities to LLMs via a lightweight, structure adapter module. Specifically, we introduce a dual-channel graph input mechanism to separate structural and semantic components for graph encoding, producing more expressive graph representations. We then incorporate a graph attention (GAT) module into LLMs' intermediate decoder layers to process structural information, enhancing the model's capability in graph-based question answering. Extensive experiments show that GRAFF significantly improves LLMs' graph-understanding ability in question answering, outperforming baselines by an average of 10.14% across four datasets. The official code for this work is available at https://github.com/hcpv/GRAFF.