Explanations of Black-Box Models based on Directional Feature Interactions

Area of Science:

• Machine Learning
• Artificial Intelligence
• Explainable AI

Background:

• Machine learning models are increasingly used, but their "black-box" nature hinders transparency.
• Current explanation methods are often univariate, focusing on individual feature importance.

Purpose of the Study:

• To extend univariate feature explanations to a higher-order bivariate approach.
• To enhance the explainability of black-box models by capturing feature interactions.

Main Methods:

• Developed a bivariate explanation method representing feature interactions as a directed graph.
• Applied the method to Shapley value explanations.
• Analyzed graph directionality to identify influential features and interchangeable feature groups.

Main Results:

• Demonstrated the ability of directional explanations to uncover feature interactions.
• Showcased the superiority of the bivariate method over state-of-the-art techniques.
• Validated the method on diverse datasets including CIFAR10, IMDB, Census, Divorce, Drug, and gene data.

Conclusions:

• Bivariate explanations offer enhanced insight into black-box model behavior.
• The directional graph analysis effectively identifies feature interactions and importance.
• This approach significantly improves model transparency and interpretability across various domains.