Exploring the Whole Rashomon Set of Sparse Decision Trees

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Machine learning algorithms typically select one model, limiting exploration of equally performing alternatives.
• The Rashomon set encompasses all models that explain data nearly equally well.
• Exploring alternative models is crucial for uncovering desirable properties not captured by standard loss functions.

Purpose of the Study:

• To introduce the first technique for completely enumerating the Rashomon set for sparse decision trees.
• To enable practitioners to explore and select from a comprehensive set of almost-optimal models.
• To provide a framework for understanding model variability and choice in machine learning.

Main Methods:

• Developed a novel technique for the complete enumeration of Rashomon sets for sparse decision trees.
• Introduced a specialized data structure for efficient querying and sampling of Rashomon sets.
• Applied the Rashomon set enumeration to analyze variable importance and derive related Rashomon sets for different metrics and data subsets.

Main Results:

• Successfully enumerated the Rashomon set for sparse decision trees, a first for highly nonlinear discrete function classes.
• Demonstrated that the Rashomon set for accuracy can be used to enumerate Rashomon sets for balanced accuracy and F1-score.
• Showcased the ability to generate Rashomon sets from data subsets using the Rashomon set of the full dataset.

Conclusions:

• The developed technique provides unprecedented control over model selection by offering all approximately equally good models.
• The Rashomon set facilitates a deeper understanding of model behavior, variable importance, and robustness across different data subsets.
• This work empowers users to choose models strategically, moving beyond the limitations of single-model outputs from standard algorithms.