Familywise Feature Importance Stability in Chemical and Materials Machine Learning

Area of Science:

• Materials Science
• Machine Learning
• Computational Chemistry

Background:

• Feature importance analysis in chemical and materials machine learning (ML) is crucial for physical insights.
• However, the robustness of feature importance rankings is rarely quantified.
• This sensitivity arises from variations in predictive models and attribution rules.

Purpose of the Study:

• To quantify the robustness of feature importance rankings across diverse ML pipelines.
• To compare different feature importance analysis families (data-driven, model-based, formula-based).
• To assess the transferability of findings across different material systems.

Main Methods:

• Compared 26 feature importance pipelines on metal-support interaction, high-entropy alloy, and halide perovskite datasets.
• Utilized data-driven, model-based, and formula-based analysis approaches.
• Examined feature importance stability across different material families and modeling assumptions.

Main Results:

• Observed high agreement within feature importance method families but substantial variance between families.
• Identified a small subset of features with stable importance across multiple material families.
• Found that mid-ranked features exhibit high family dependence, shifting importance based on modeling assumptions.

Conclusions:

• Robust interpretability in materials ML requires quantifying feature importance robustness.
• Feature importance should be reported by method family or correlation-based clusters.
• Supplementing importance rankings with resampling intervals enhances reliability.