Multi-level encoder architectures for event causality identification

Event causality identification (ECI) aims to determine whether two events have causality, which is highly dependent on the context. Therefore, it is crucial to effectively encode the contextual information of events within lengthy documents for achieving optimal performance. Recent works have focused on introducing context-independent external knowledge bases and model inference components while slighting exploration of event contexts, especially the interactions between event pairs and the critical role of the main event in the document. To fully utilize this knowledge, we found it helpful to use a multi-level structure to encode the context. The multi-level structure enables us to capture information at various levels within a lengthy document, employing approaches that are tailored to each level. In this paper, we propose Multi-level Encoder Architectures (MEA) capable of adequately encoding different levels of abundant information. In sentence-level, we use a pre-trained language model to enrich the event representation. In event-level, we introduce an innovative approach to link all events forming an event graph that leverages the pivotal role of the main event, then encode these unstructured relations with graph neural networks. In event-pair-level and discourse-level, we employ a self-attention mechanism to capture the uncertain relationships from event-pair to event-pair and sentence to sentence. Comprehensive experiments on two widely used public datasets demonstrate the effectiveness of the proposed methods at each level and the overall framework.¹.