Distilling the Knowledge of Ensembles for Uncertainty-Aware Genomic Deep Learning

While deep neural networks have revolutionized predictive modeling in regulatory genomics, challenges persist in ensuring the reliability of their predictions and understanding model decision-making processes. This chapter introduces DEGU (Distilling Ensembles for Genomic Uncertainty-aware models), a model-training framework that integrates ensemble learning with knowledge distillation to enhance the robustness and explainability of genomic deep learning predictions. DEGU distills the distribution of predictions from an ensemble of models into a single student model, capturing both the ensemble's average predictions and the variability across models, which represents epistemic uncertainty. Additionally, DEGU can estimate aleatoric uncertainty by modeling variability across experimental replicates. Through applications across diverse functional genomic prediction tasks, we demonstrate that DEGU-trained models achieve ensemble-level performance in a single model, exhibit improved generalization to out-of-distribution sequences, provide more consistent mechanistic explanations through attribution analysis, and generate calibrated uncertainty estimates. These advances position DEGU as a practical framework for developing trustworthy and robust deep learning applications in genomics research.