Extreme image transformations affect humans and machines differently.

Area of Science:

• Computational Neuroscience
• Computer Vision
• Cognitive Science

Background:

• Recent artificial neural networks (ANNs) mimic primate neural and human performance in object recognition.
• ANNs often rely on low-level features, making them vulnerable to out-of-distribution or adversarial inputs.
• Human visual processing excels at abstract pattern recognition, demonstrating resilience to extreme image distortions.

Purpose of the Study:

• To introduce novel image transforms inspired by neurophysiological findings.
• To evaluate and compare the object recognition performance of humans and ANNs using these transforms.
• To identify specific transforms that challenge ANNs compared to human visual capabilities.

Main Methods:

• Development of a novel set of image transforms based on neurophysiological principles.
• Comparative evaluation of human and ANN performance on an object recognition task utilizing these transforms.
• Quantification of accuracy differences and creation of a difficulty ranking for transforms based on human data.

Main Results:

• ANNs outperformed humans on certain image transforms.
• Humans significantly outperformed ANNs on other transforms that were easy for human perception.
• A clear ranking of transform difficulty for human visual processing was established.

Conclusions:

• Human visual processing and current ANNs differ fundamentally in how they handle visual information.
• ANNs exhibit limitations in generalizing abstract patterns, unlike human vision.
• Insights from human visual processing can guide the development of more robust and adaptable ANNs.