Multi-Hop Knowledge Graph Reasoning with Pre-trained Models and Continuous Rule Guidance

The multi-hop reasoning tasks have been formalized as a sequential decision problem, which rely on multiple walk-based agents (the entity agent and the relation agent) to extend reasoning paths, until the target answer is reached. Knowledge graph embedding models and symbol logical rules are adopted to guide the entity and the relation agents, respectively. However, in an incomplete knowledge graph environment, this formalization will be faced with two main difficulties: (1) how to choose the next step appropriately for both the entity and the relation agents, and (2) the sparse reward signals received by the relation agent, due to the discrete nature of symbolic logical rules. To conquer these two challenges, we propose a novel Pre-trained Transformer Model for Multi-hop Reasoning approach named PTM4MR. Firstly, inspired by the NSP training strategy in pre-trained language model such as BERT, we introduce three pre-trained tasks, such as next action prediction, next entity prediction, and next relation prediction, to enhance the perception ability of multiple agents for next step selection, i.e., to improve their sensitivity for path exploration. Secondly, we propose a relational argument embedding based schema to estimate a numerical and continuous reward, which can alleviate the problem of sparse reward signals caused by the limitation of symbolic logical rules. With extensive experiments on several benchmark datasets, our proposed approach PTM4MR achieves considerable performance improvements compared with state-of-the-art baselines. The source code is publicly available at https://github.com/WeidongLi-KG/PTM4MR.