Benchmarking interpretability of deep learning for predictive genomics: Recall, precision, and variability of feature...

Area of Science:

• Genetics and Bioinformatics
• Machine Learning in Biology

Background:

• Deep neural networks (DNNs) can model complex polygenic traits.
• Reliability of DNN interpretability methods for identifying genetic variants is uncertain.
• Accurate identification of genetic variants is crucial for understanding trait architecture.

Purpose of the Study:

• To introduce and apply a benchmarking framework to quantify DNN interpretability for genetic variant identification.
• To evaluate the performance of four attribution algorithms (Saliency, Gradient SHAP, DeepLIFT, Integrated Gradients) in identifying causal variants for standing height prediction.
• To assess the impact of SmoothGrad noise averaging on attribution performance and stability.

Main Methods:

• Developed a benchmarking framework to measure attribution recall, precision, and stability.
• Trained feed-forward neural networks on UK Biobank genotype data (500K+ variants, 300K participants) for standing height prediction.
• Evaluated four attribution algorithms with and without SmoothGrad, using synthetic spike-in and null decoy variants to assess performance.

Main Results:

• SmoothGrad improved average attribution recall by ~0.16 and precision by ~0.06 at the top 1% threshold.
• Attribution stability was comparable across methods, with median relative standard deviations of 0.4-0.5.
• Saliency achieved the highest composite score, demonstrating the best balance of recall, precision, and stability among tested methods.

Conclusions:

• The developed benchmarking framework effectively quantifies DNN interpretability for genetic variant identification.
• Saliency attribution offers a reliable method for identifying genetic variants contributing to polygenic traits using deep learning.
• Findings provide a foundation for more trustworthy application of deep learning in genetic research.