ChromBERT: A foundation model for learning interpretable representations for context-specific transcriptional...

Area of Science:

• Genomics
• Computational Biology
• Molecular Biology

Background:

• Transcriptional regulatory networks (TRNs) govern gene expression through context-specific interactions.
• Sparse cell-type-specific chromatin immunoprecipitation sequencing (ChIP-seq) data limits understanding of TRNs.
• Existing methods struggle with the challenge of incomplete regulatory data.

Purpose of the Study:

• To develop a foundation model for deciphering context-specific TRNs.
• To overcome data sparsity in ChIP-seq profiles for transcription regulators.
• To generate interpretable TRN representations and infer regulatory roles.

Main Methods:

• Pre-training a foundation model (ChromBERT) on large-scale human ChIP-seq datasets (~1,000 regulators).
• Utilizing prompt-enhanced fine-tuning for improved imputation of unseen cistromes.
• Applying lightweight fine-tuning for cell-type-specific downstream tasks.

Main Results:

• ChromBERT learns genome-wide regulatory cooperation syntax.
• The model outperforms existing methods in imputing sparse cistrome data.
• Fine-tuned representations capture cell-type-specific regulatory effects and dynamics.
• Inferred regulatory roles of transcription regulators are identified without new ChIP-seq data.

Conclusions:

• ChromBERT effectively models and interprets transcriptional regulation across diverse biological contexts.
• The foundation model addresses the critical limitation of sparse transcription regulator data.
• This approach significantly enhances the ability to study context-specific TRNs.