Neural Encoding and Decoding at Scale.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computational Biology

Background:

• Large-scale, multi-animal models are crucial for understanding neural activity and behavior relationships.
• Current models are limited as they focus on either encoding (neural activity from behavior) or decoding (behavior from neural activity), not both.

Purpose of the Study:

• To introduce a multimodal, multi-task model for simultaneous Neural Encoding and Decoding at Scale (NEDS).
• To capture the bidirectional relationship between neural activity and behavior more effectively.

Main Methods:

• Developed a novel multi-task-masking strategy involving neural, behavioral, within-modality, and cross-modality masking.
• Pretrained the NEDS model on the International Brain Laboratory (IBL) repeated site dataset (83 animals).
• Fine-tuned the pretrained model on new animals to evaluate performance.

Main Results:

• NEDS achieved state-of-the-art performance in both encoding and decoding tasks when pretrained on multi-animal data.
• Learned embeddings from NEDS showed emergent properties, accurately predicting brain regions without explicit training.
• Demonstrated superior performance compared to existing large-scale models.

Conclusions:

• NEDS represents a significant advancement in modeling the brain, enabling seamless translation between neural activity and behavior.
• The model's emergent properties suggest potential for broader applications in neuroscience research.
• NEDS is a step towards creating a foundational model of the brain.