Generating Biomedical Hypothesis With Spatiotemporal Transformers

Generating biomedical hypotheses is a difficult task as it requires uncovering the implicit associations between massive scientific terms from a large body of published literature. A recent line of Hypothesis Generation (HG) approaches - temporal graph-based approaches - have shown great success in modeling temporal evolution of term-pair relationships. However, these approaches model the temporal evolution of each term or term-pair with Recurrent Neural Network (RNN) independently, which neglects the rich covariation among all terms or term-pairs while ignoring direct dependencies between any two timesteps in a temporal sequence. To address this problem, we propose a Spatiotemporal Transformer-based Hypothesis Generation (STHG) method to interleave spatial covariation and temporal progression in a unified framework for constructing direct connections between any two term-pairs while modeling the temporal relevance between any two timesteps. Experiments on three biomedical relationship datasets show that STHG outperforms the state-of-the-art methods.