An interpretable block-attention network for identifying regulatory feature interactions

The importance of regulatory features in health and disease is increasing, making it crucial to identify the hallmarks of these features. Self-attention networks (SAN) have given rise to numerous models for the prediction of complex phenomena. But the potential of SANs in biological models was limited because of high memory requirement proportional to input token length and lack of interpretability of self-attention scores. To overcome these constraints, we propose a deep learning model named Interpretable Self-Attention Network for REGulatory interactions (ISANREG) that combines both block self-attention and attention-attribution mechanisms. This model predicts transcription factor-bound motif instances and DNA-mediated TF-TF interactions using self-attention attribution scores derived from the network, overcoming the limitations of previous deep learning models. ISANREG will serve as a framework for other biological models in interpreting the contribution of the input with single-nucleotide resolution.