Gaussian Process Regression With Interpretable Sample-Wise Feature Weights

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Gaussian Process Regression (GPR) is a powerful machine learning tool known for accurate predictions and uncertainty quantification.
• A key limitation of GPR is its lack of interpretability regarding feature contributions to predictions.

Purpose of the Study:

• To develop an interpretable version of GPR that reveals feature contributions for individual predictions.
• To maintain the high predictive performance characteristic of GPR.

Main Methods:

• Proposed GPR with Local Explanation (GPR-LE), integrating an interpretable locally linear model for prediction and explanation.
• Utilized multivariate Gaussian process priors for weight vectors and maximized marginal likelihood for hyperparameter estimation.
• Derived closed-form solutions for predicting target variables, weight vectors, and their uncertainties for new samples.

Main Results:

• GPR-LE achieved predictive performance comparable to standard GPR on benchmark datasets.
• The proposed model demonstrated superior interpretability compared to existing interpretable models, both quantitatively and qualitatively.
• GPR-LE successfully revealed feature contributions for each sample's prediction.

Conclusions:

• GPR-LE offers a viable solution for enhancing the interpretability of Gaussian Process Regression without sacrificing predictive accuracy.
• The model provides valuable insights into feature importance at a local, sample-specific level.
• This advancement opens new possibilities for applying GPR in domains requiring transparent decision-making.