Learning and interpreting the gene regulatory grammar in a deep learning framework.

Area of Science:

• Computational Biology and Bioinformatics
• Genomics and Gene Regulation

Background:

• Deep neural networks (DNNs) excel at identifying gene regulatory sequences but offer limited biological insight due to interpretation challenges.
• Understanding the combinatorial binding of transcription factors (TFBS), or regulatory grammar, is crucial for enhancer activity, yet its prevalence across different sequence architectures remains unclear.

Purpose of the Study:

• To investigate the capacity of DNNs to learn enhancer regulatory grammar through hypothesis-driven analyses.
• To develop and validate a method for interpreting DNNs to extract learned regulatory patterns.

Main Methods:

• Creation of synthetic datasets modeling various TFBS patterns (homotypic/heterotypic clusters, enhanceosomes) using real TF motifs.
• Training deep residual neural networks (ResNets) on these datasets for multi-label regulatory sequence prediction.
• Development of a gradient-based unsupervised clustering method to extract patterns from ResNet models.

Main Results:

• Simulated regulatory grammars are primarily learned in the penultimate layer of ResNets.
• The developed method accurately retrieves regulatory grammar from synthetic data, even with noisy or heterogeneous enhancer categories.
• Identified specific scenarios where ResNets struggle to learn simulated regulatory grammars and successfully applied the method to identify a known TF cluster in mouse enhancers.

Conclusions:

• The study provides a framework for interpreting regulatory rules learned by ResNets.
• The effectiveness of ResNets in learning regulatory grammar is contingent on the specific prediction task and sequence architecture.