Unmasking Clever Hans predictors and assessing what machines really learn

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Modern learning machines achieve high accuracy on complex tasks, exhibiting intelligent behavior.
• Existing performance metrics may not differentiate diverse problem-solving strategies.
• There is a need for nuanced evaluation of machine intelligence.

Purpose of the Study:

• To apply explainability techniques to analyze machine learning model behavior.
• To introduce Spectral Relevance Analysis (SRA) for characterizing nonlinear learning machines.
• To provide a framework for more critical assessment of machine intelligence.

Main Methods:

• Analysis of state-of-the-art learning machines on computer vision and arcade game tasks.
• Application of recent explainability techniques to decision-making processes.
• Development and implementation of semi-automated Spectral Relevance Analysis (SRA).

Main Results:

• Observed a spectrum of machine problem-solving behaviors, from naive to strategic.
• Demonstrated that standard evaluation metrics are insufficient for distinguishing these behaviors.
• Validated SRA as an effective method for characterizing and validating nonlinear learning machine behavior.

Conclusions:

• Spectral Relevance Analysis offers a practical approach to assess if learned models perform as intended.
• A cautious and nuanced approach is needed to evaluate the successes of machine intelligence.
• Reliable validation of machine learning models requires methods beyond standard performance metrics.