A self-attention model for inferring cooperativity between regulatory features

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Deep learning models excel at predicting biological processes like gene expression.
• Extracting biologically relevant insights from these models is crucial but challenging.
• Identifying cooperativity between regulatory elements, key to gene expression, remains an open problem.

Purpose of the Study:

• To introduce SATORI, a novel self-attention-based model for detecting regulatory element interactions.
• To capture global interactions between regulatory elements within a sequence.
• To enhance the interpretability of deep learning models in genomics.

Main Methods:

• Utilized a combination of convolutional layers and a self-attention mechanism.
• Developed a self-attention-based model named SATORI (Self-ATtentiOn based model to detect Regulatory element Interactions).
• Focused on detecting transcription factor-transcription factor (TF-TF) interactions.

Main Results:

• SATORI successfully identified numerous statistically significant TF-TF interactions, many previously documented.
• The model demonstrated superior performance in detecting experimentally verified TF-TF interactions compared to existing methods.
• SATORI does not require computationally intensive post-processing steps.

Conclusions:

• SATORI offers an effective and efficient approach for detecting regulatory element interactions, particularly TF-TF interactions.
• The model's self-attention mechanism facilitates a comprehensive understanding of element cooperativity.
• The SATORI framework is adaptable for detecting various feature interactions in attention-based models.