Task-dependent color representation in convolutional neural networks.

Area of Science:

• Computational neuroscience
• Machine learning
• Visual perception

Background:

• Convolutional neural networks (CNNs) exhibit color-selective units.
• Human color perception is task-dependent.
• Representational dissimilarity matrices (RDMs) and multidimensional scaling (MDS) can visualize color representations.

Purpose of the Study:

• To investigate how task variations influence color representations in CNNs.
• To compare CNN color representations with human visual processing.
• To explore the development of color representations in networks trained on color-relevant and color-irrelevant tasks.

Main Methods:

• Training CNNs on chromatic stimuli with varied tasks (color categorization, appearance rating, luminance/spatial judgments).
• Analyzing network layers using RDMs and MDS to create geometric color spaces.
• Comparing color representations across different task conditions and network layers.

Main Results:

• CNN color representations differed significantly based on task conditions.
• Structured color representations emerged even for tasks initially deemed color-irrelevant.
• Color representations diverged across network layers, showing differences even near the input layer.
• Network variance was lowest for color-relevant tasks.

Conclusions:

• Task demands critically shape color representations within CNNs.
• CNNs can learn to leverage color cues for tasks not explicitly requiring them, mirroring potential human brain mechanisms.
• These findings provide a model for generating hypotheses about task-dependent color processing in the human brain that can be tested with neuroimaging.