The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance

Area of Science:

• Computational biology
• Statistical modeling
• Genetics

Background:

• Variable importance quantification is crucial in high-stakes fields like genetics and medicine.
• Existing methods often yield conflicting conclusions due to reliance on single models and datasets.
• Lack of generalizability arises because not all good explanations are stable across data perturbations.

Approach:

• Propose a novel variable importance framework quantifying importance across all good models and stable data distributions.
• Framework is flexible and integrates with existing model classes and global variable importance metrics.
• Provide theoretical guarantees on estimator consistency and finite sample error rates.

Key Points:

• The framework recovers variable importance rankings in complex simulations where other methods fail.
• It accurately estimates the true importance of a variable for the underlying data distribution.
• A real-world case study identified a novel gene associated with predicting HIV load.

Conclusions:

• The proposed framework offers a more robust and generalizable approach to variable importance.
• It resolves ambiguities arising from multiple equally valid models.
• Enables more reliable insights in genetics, medicine, and public policy.