Parameter-efficient fine-tuning enables scalable transfer of regulatory sequence models to novel contexts

DNA sequence deep learning models can accurately predict epigenetic and transcriptional profiles, enabling analysis of gene regulation and genetic variant effects. While large-scale models like Enformer and Borzoi are trained on abundant data, they cannot cover all cell states and assays, necessitating training new model to analyze gene regulation in novel contexts. However, training models from scratch for new datasets is computationally expensive.

In this study, we systematically develop and evaluate a transfer learning framework based on parameter-efficient fine-tuning for supervised regulatory sequence models. Using the state-of-the-art model Borzoi, our framework enables accurate model transfer while significantly reducing runtime and memory requirements. Across bulk and single cell RNA-seq datasets, the transferred models effectively predict held-out gene expression changes, identify regulatory drivers in perturbation conditions, and predict cell-type-specific variant effects. We further demonstrate that transferring Borzoi to relevant cell types facilitates mechanistic interpretation of fine-mapped GWAS variants.

Our framework offers a scalable and practical solution for extending large sequence models to novel biological contexts, enabling mechanistic insight into gene regulation and variant effects.