Associative attention networks for temporal relation extraction from electronic health records

Temporal relations are crucial in constructing a timeline over the course of clinical care, which can help medical practitioners and researchers track the progression of diseases, treatments and adverse reactions over time. Due to the rapid adoption of Electronic Health Records (EHRs) and high cost of manual curation, using Natural Language Processing (NLP) to extract temporal relations automatically has become a promising approach. Typically temporal relation extraction is formulated as a classification problem for the instances of entity pairs, which relies on the information hidden in context. However, EHRs contain an overwhelming amount of entities and a large number of entity pairs gathering in the same context, making it difficult to distinguish instances and identify relevant contextual information for a specific entity pair. All these pose significant challenges towards temporal relation extraction while existing methods rarely pay attention to. In this work, we propose the associative attention networks to address these issues. Each instance is first carved into three segments according to the entity pair to obtain the differentiated representation initially. Then we devise the associative attention mechanism for a further distinction by emphasizing the relevant information, and meanwhile, for the reconstruction of association among segments as the final representation of the whole instance. In addition, position weights are utilized to enhance the performance. We validate the merit of our method on the widely used THYME corpus and achieve an average F1-score of 64.3% over three runs, which outperforms the state-of-the-art by 1.5%.