Interpreting Neural Networks for Biological Sequences by Learning Stochastic Masks

Area of Science:

• Computational Biology and Bioinformatics
• Machine Learning in Genomics
• Structural Bioinformatics

Background:

• Sequence-based neural networks excel at biological data prediction but lack interpretability.
• Existing feature attribution methods struggle with discrete molecular sequences and non-linear interactions.
• Need for advanced methods to interpret complex biological sequence data.

Purpose of the Study:

• To develop a deep learning-based interpretation method for molecular sequences.
• To address limitations of current feature attribution techniques for discrete biological data.
• To enable accurate identification of salient sequence positions and their biological relevance.

Main Methods:

• Introduced Scrambler networks, a deep learning approach inspired by computer vision and NLP.
• Scramblers utilize learned input masks to identify salient sequence positions.
• Developed a mechanism to predict Position-Specific Scoring Matrices (PSSMs) by increasing entropy at unimportant positions.

Main Results:

• Scramblers successfully interpreted genetic variant effects and cis-regulatory element interactions.
• The method explained binding specificity in protein-protein interactions and identified structural determinants in designed proteins.
• Demonstrated efficient attribution across large datasets with high-quality explanations, outperforming existing methods.

Conclusions:

• Scrambler networks provide an effective solution for interpreting complex biological sequence data.
• The approach enhances understanding of molecular sequence function and interactions.
• Scramblers represent a significant advancement in applying deep learning for biological sequence interpretation.