Stimulus symmetries can confound representational similarity analyses.

Area of Science:

• Computational neuroscience
• Machine learning theory
• Neural coding

Background:

• Representational Similarity Analysis (RSA) is a popular method for characterizing neural codes.
• Understanding the properties and limitations of Representational Similarity Matrices (RSMs) is crucial for accurate interpretation.
• Existing methods may not fully account for complex relationships between network inputs and representations.

Purpose of the Study:

• To investigate how symmetries in network inputs can influence Representational Similarity Matrices (RSMs).
• To explore the impact of different training methods on neural code properties and their corresponding RSMs.
• To highlight challenges in comparing nonlinear neural codes.

Main Methods:

• Analysis of representational geometries derived from neural network models.
• Simulations using stochastic gradient descent and energetic regularization.
• Examination of networks trained on image data with latent symmetries.

Main Results:

• Input symmetries can lead to functionally equivalent representations with distinct RSMs, indicating different representational geometries.
• Sparse and drifting codes can emerge from training methods like stochastic gradient descent or energetic regularization, resulting in drifting RSMs.
• These phenomena were observed in networks trained for image encoding, even with latent symmetries.

Conclusions:

• Symmetries in stimuli pose a significant challenge for RSM-based analyses of neural codes.
• Functionally equivalent representations are not always related by simple rotations, complicating comparisons.
• Careful consideration of input symmetries and training dynamics is necessary for robust interpretation of neural coding.