Can deep convolutional neural networks support relational reasoning in the same-different task?

Area of Science:

• Artificial Intelligence
• Computer Vision
• Cognitive Science

Background:

• Same-different visual reasoning is crucial for abstract thought.
• Deep convolutional neural networks (DCNNs) are increasingly tested for this skill, but results are debated.
• Previous studies often used training and testing data from the same pixel distribution, limiting conclusions.

Purpose of the Study:

• To investigate the capacity of DCNNs for relational same-different visual reasoning.
• To determine if DCNNs can generalize same-different classification beyond pixel-level similarities.
• To assess the robustness of DCNNs and relation networks when faced with distribution shifts.

Main Methods:

• Conducted simulations using ResNet-based DCNN architectures.
• Tested DCNN performance on same-different classification with varying pixel-level distributions between training and testing data.
• Evaluated a relation network architecture on the same tasks.
• Included expanded training regimes and multitask learning contexts.

Main Results:

• ResNet models achieved same-different classification accuracy only when test images matched training images at the pixel level.
• Performance substantially decreased when the testing distribution shifted, even without altering object relations.
• These limitations persisted across expanded training strategies and multitask learning.
• Relation networks exhibited similar constraints.

Conclusions:

• Current DCNNs, including ResNet and relation networks, struggle with generalizing same-different visual reasoning beyond pixel-level similarities.
• Learning abstract relational concepts appears to be beyond the scope of existing DCNN architectures.
• Further research is needed to develop models capable of robust visual relational reasoning.