Temporal attention networks for biomedical hypothesis generation.

Area of Science:

• Biomedical Informatics
• Computational Biology
• Artificial Intelligence

Background:

• Hypothesis Generation (HG) uncovers hidden scientific term associations crucial for public health innovations.
• Recurrent Neural Networks (RNNs) have been used for HG, but struggle with complex spatiotemporal dependencies.
• Attention mechanisms offer a promising alternative for modeling temporal evolution in term-pair relations.

Purpose of the Study:

• To develop a novel method for accurately modeling the temporal evolution of biomedical term-pair relations.
• To capture crucial spatiotemporal dependencies for inferring future scientific connections.
• To enhance Hypothesis Generation (HG) using pure attention mechanisms.

Main Methods:

• Proposed Temporal Attention Networks (TAN) for Biomedical Hypothesis Generation.
• Formulated HG as a future connectivity prediction task in a temporal attributed graph.
• Developed Temporal Spatial Attention Module (TSAM) for smoothing embeddings and Temporal Difference Attention Module (TDAM) for highlighting historical changes.

Main Results:

• TAN significantly outperformed baseline methods on three real-world biomedical datasets (Immunotherapy, Virology, Neurology).
• Achieved Micro-F1 Score improvements of 12.03% (Immunotherapy), 4.59% (Virology), and 2.34% (Neurology).
• Demonstrated TAN's ability to model complex spatiotemporal dependencies and capture temporal relation evolution.

Conclusions:

• Introduced TAN, a novel attention-based model for learning spatiotemporal embeddings for HG.
• TAN effectively models relationship evolution by considering both continuity and difference in temporal embeddings.
• The attention mechanism in TAN extracts critical spatiotemporal dependencies for hypothesis generation.