Bayesian subset selection and variable importance for interpretable prediction and classification.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Classical subset selection methods suffer from instability and lack of regularization.
• Interpretable learning, scientific discovery, and data compression benefit from subset selection.
• Bayesian methods offer a robust framework to address limitations of traditional subset selection.

Purpose of the Study:

• To develop a Bayesian framework for extracting families of near-optimal variable subsets.
• To provide a new pathway for model interpretation and variable importance metrics.
• To derive optimal linear coefficients for any subset, incorporating regularization and uncertainty quantification.

Main Methods:

• Utilizing a Bayesian predictive model to extract an 'acceptable family' of subsets.
• Applying Bayesian decision analysis to derive optimal linear coefficients for selected subsets.
• Evaluating the approach on simulated and real-world data, including a large education dataset.

Main Results:

• The proposed Bayesian subset selection demonstrates superior prediction, interval estimation, and variable selection compared to existing methods.
• Identified over 200 distinct subsets with near-optimal out-of-sample predictive accuracy on an education dataset.
• Developed novel metrics for variable importance based on subset inclusion.

Conclusions:

• The Bayesian framework effectively addresses challenges in classical subset selection, offering stability and interpretability.
• The 'acceptable family' concept and derived metrics provide enhanced model understanding.
• The approach yields significant improvements in predictive performance and variable selection for complex datasets.