Trying to outrun causality with machine learning: Limitations of model explainability techniques for exploratory...

Area of Science:

• Psychology
• Computer Science
• Data Science

Background:

• Machine learning explainability techniques offer psychologists a way to interrogate models and understand phenomena.
• Researchers may use these techniques in exploratory studies to avoid restrictive functional forms, aiming to identify predictive variables.

Purpose of the Study:

• To demonstrate how machine learning algorithms' sensitivity to causal data structures can affect the perceived importance of predictors.
• To highlight that apparent unimportance identified by explainability techniques may stem from regression's mathematical properties and causal independencies, not technique limitations.

Main Methods:

• The study analyzes the behavior of machine learning algorithms in the context of underlying causal structures in data.
• It investigates the interaction between regression's mathematical implications and conditional independencies within causal structures.

Main Results:

• Machine learning algorithms are highly sensitive to the data's underlying causal structure.
• Explainability techniques may incorrectly deem important predictors as unimportant due to this sensitivity and regression properties.

Conclusions:

• The findings underscore that apparent unimportance of predictors is a consequence of regression mathematics and causal structure, not solely a limitation of explainability techniques.
• Alternative recommendations are provided for psychologists seeking effective data exploration methods to identify significant variables.