Unboxing deep learning models using gradient-based methods - A tutorial.

Area of Science:

• Spectroscopy
• Machine Learning
• Chemometrics

Background:

• Deep learning models, especially convolutional neural networks, excel in spectroscopic data analysis.
• The "black-box" nature of these models hinders their practical application and validation.
• Understanding predictor contributions is crucial for model interpretability.

Purpose of the Study:

• To provide a tutorial on gradient-based methods for explaining deep learning models in spectroscopy.
• To demonstrate how these methods assess variable importance and model validity.
• To introduce and illustrate integrated gradients and a novel gradient-response product (GIR) modification.

Main Methods:

• Detailed derivations of saliency maps (vanilla gradients) and integrated gradients.
• Step-by-step numerical examples for computational procedures.
• Application to simulated and real-world spectroscopic datasets (Tecator, Beer).
• Implementation guidance using Python and Jupyter notebooks.

Main Results:

• Integrated gradients yield stable and interpretable importance scores.
• Scores are consistent across various model architectures and training stages.
• The GIR modification enhances informativeness for noisy or systematically biased data.
• Gradient-based methods effectively evaluate variable importance and model validity.

Conclusions:

• Gradient-based methods, particularly integrated gradients, significantly improve the interpretability of deep learning models for spectroscopic data.
• These techniques offer reliable variable importance assessment, crucial for scientific validation.
• The provided tutorial and code facilitate practical implementation and broader adoption.