A Benchmark for Compositional Visual Reasoning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Cognitive Science

Background:

• Human vision efficiently parses complex scenes and object relations.
• AI visual reasoning benchmarks show progress, but lag in sample efficiency.
• Human learning efficiency is linked to harnessing compositionality.

Purpose of the Study:

• Introduce a novel visual reasoning benchmark, Compositional Visual Relations (CVR).
• Drive progress in developing more data-efficient AI learning algorithms.
• Evaluate AI's ability to learn and generalize visual reasoning tasks.

Main Methods:

• Inspired by fluid intelligence and non-verbal reasoning tests.
• Developed a novel method for composing abstract rules and generating large-scale image datasets.
• Benchmark includes measures for sample efficiency, generalization, compositionality, and transfer learning.

Main Results:

• Convolutional architectures outperformed transformer-based architectures across most measures.
• All computational models demonstrated significantly lower data efficiency than humans.
• Self-supervised learning of visual representations did not bridge the data efficiency gap.

Conclusions:

• The Compositional Visual Relations (CVR) benchmark facilitates research into data-efficient AI.
• Convolutional neural networks show advantages over transformers in this visual reasoning context.
• Further research is needed to develop AI that can harness compositionality for efficient learning.