Contextual associations represented both in neural networks and human behavior

Area of Science:

• Computer Vision
• Cognitive Science
• Artificial Intelligence

Background:

• Contextual associations are vital for human object recognition.
• The role and characteristics of context in artificial vision systems, like convolutional neural networks (CNNs), are not well understood.

Purpose of the Study:

• To investigate if CNNs represent contextually related objects (e.g., bicycle-helmet) more similarly than unrelated objects (e.g., bicycle-fork).
• To determine if CNN representations of context align with human behavioral assessments of contextual relatedness.

Main Methods:

• Utilized a diverse set of 73 contexts with objects presented against a white background.
• Analyzed representations of contextually related and unrelated object pairs within multiple layers of CNNs.
• Correlated CNN similarity measures with human behavioral data from experiments on contextual relatedness.

Main Results:

• CNNs demonstrated more similar representations for contextually related objects compared to unrelated objects across all layers.
• The similarity in CNN representations significantly correlated with human behavioral judgments, particularly in the later network layers.
• Contextual regularities are inherently learned and represented in CNNs trained for object recognition.

Conclusions:

• CNNs inherently represent contextual information due to object recognition training.
• Later layers of CNNs capture contextual regularities that align with human perception and behavior.
• This research bridges the understanding of context in human and artificial vision systems.