Explainability in transformer models for functional genomics.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Deep learning excels at automated feature extraction from raw data, crucial for functional genomics.
• Identifying relevant nucleotide motifs in DNA sequences is key for understanding gene regulation.
• Developing explainable AI methods is essential for interpreting complex biological models.

Purpose of the Study:

• To present a novel approach for gaining insights into the transcription process using deep learning.
• To apply a transformer-based neural network for prokaryotic genome annotation.
• To investigate the decision-making process of trained models in functional genomics.

Main Methods:

• Utilized a transformer-based neural network framework for analyzing prokaryotic genomes.
• Applied automated feature extraction to DNA sequences.
• Analyzed the specialization of model subunits (attention heads) for biological insight.

Main Results:

• The transformer model successfully identified transcription factors and their binding sites in Escherichia coli.
• Attention heads within the model demonstrated specialization for recognizing consensus sequences.
• The approach uncovered both known and potentially novel regulatory elements involved in transcription initiation.

Conclusions:

• Transformer models can automatically specialize attention heads for identifying key genomic elements.
• This work highlights the potential of transformer networks for creating explainable AI in genomics.
• The findings offer valuable insights into the bacterial transcription process and regulatory mechanisms.