How 'Neural' is a Neural Foundation Model?

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Systems Neuroscience

Background:

• Foundation models excel at modeling biological visual systems.
• The black-box nature of these models hinders understanding of brain function.
• Physiological analysis of neural activity models is crucial for biological relevance.

Purpose of the Study:

• To investigate the internal mechanisms of a state-of-the-art foundation model for neural activity.
• To characterize individual 'neurons' based on temporal response properties to stimuli.
• To understand how different stimuli and neurons are represented within the model's architecture.

Main Methods:

• Characterized model 'neurons' by their temporal response properties to parametric stimuli.
• Built decoding and neural encoding manifolds to analyze stimulus and neural representations.
• Introduced a 'tubularity' metric to quantify stimulus-dependent neural activity development.

Main Results:

• Different model modules (encoder, recurrent, readout) exhibit distinct representational structures.
• The recurrent module enhances representation by differentiating temporal stimulus patterns.
• The readout module achieves high fidelity via specialized feature maps, lacking biological plausibility.
• The 'tubularity' metric indicates biologically plausible neural activity development.

Conclusions:

• The study provides insights into the internal workings and biological relevance of neural foundation models.
• Findings suggest design modifications for greater biological alignment: incorporating early recurrence and constrained readout features.
• This work bridges artificial intelligence and neuroscience by analyzing model internals through a physiological lens.