Cross-validated permutation feature importance considering correlation between features.

Area of Science:

• Computational chemistry
• Data science
• Machine learning

Background:

• Accurate model interpretation is crucial in molecular, material, and process design.
• Permutation Feature Importance (PFI) is a common method but suffers from instability with low sample sizes and correlated features.

Purpose of the Study:

• To propose a novel method, Cross-Validated Permutation Feature Importance (CVPFI), for robust feature importance evaluation.
• To address the limitations of PFI in scenarios with small datasets and multicollinearity.

Main Methods:

• Developed CVPFI by integrating cross-validation into the PFI calculation.
• Incorporated absolute correlation coefficients to handle strongly correlated features.
• Validated CVPFI using numerical simulation and actual compound data.

Main Results:

• CVPFI demonstrates stable and appropriate feature importance evaluation across various conditions.
• The method performs well with low sample sizes, mixed linear/nonlinear relationships, and correlated/quantized/biased features.
• Case studies confirmed CVPFI's superiority over traditional PFI.

Conclusions:

• CVPFI provides a more reliable approach to feature importance assessment in complex datasets.
• This method enhances model interpretability in scientific and engineering design processes.
• The proposed CVPFI method is publicly available via Python code.