Selecting deep neural networks that yield consistent attribution-based interpretations for genomics.

Area of Science:

• Genomics
• Computational Biology
• Machine Learning

Background:

• Deep neural networks (DNNs) are powerful tools for predicting molecular activities from DNA sequences.
• Attribution analysis in DNNs can reveal important sequence motifs but often yields spurious importance scores.
• Standard model selection based on validation performance does not guarantee reliable DNN explanations.

Purpose of the Study:

• To introduce novel methods for quantifying feature consistency across DNN attribution maps.
• To develop a multivariate model selection framework integrating generalization performance and explanation reliability.
• To identify DNNs that provide both high predictive accuracy and interpretable biological insights.

Main Methods:

• Developed two approaches to quantify the consistency of important features across multiple attribution maps.
• Integrated consistency metrics into a multivariate model selection framework.
• Evaluated the approach using synthetic data and real-world chromatin accessibility data.

Main Results:

• The proposed consistency metrics reflect a qualitative property of human-interpretable attribution maps.
• The multivariate model selection framework successfully identified models with high generalization and reliable explanations.
• Demonstrated the efficacy of the approach across various DNNs and datasets.

Conclusions:

• Model selection for DNNs in genomics should consider explanation reliability alongside predictive performance.
• Feature consistency across attribution maps is a valuable metric for assessing DNN interpretability.
• This framework enhances the trustworthiness of DNNs for biological sequence analysis.