Modeling conditional distributions of neural and behavioral data with masked variational autoencoders

Area of Science:

• Computational Neuroscience
• Machine Learning in Neuroscience
• Data Analysis in Neuroscience

Background:

• Understanding the relationship between neural activity and behavior is crucial in neuroscience.
• Existing methods like encoding/decoding models and dimensionality reduction techniques have limitations in modeling complex conditional distributions.
• Variational Autoencoders (VAEs) are powerful for dimensionality reduction but struggle with arbitrary conditional distributions, especially simultaneously.

Purpose of the Study:

• To present a unified Variational Autoencoder (VAE)-based approach for calculating conditional distributions in neuroscience.
• To enable simultaneous dimensionality reduction and accurate modeling of neural encoding and decoding distributions.
• To scale common neuroscience analyses to high-dimensional, multi-modal datasets.

Main Methods:

• Developed a novel VAE-based framework to infer conditional distributions between neural activity and behavior.
• Validated the approach on a task with known ground truth.
• Applied the method to analyze masked body parts in walking flies and decode motor trajectories in a monkey-reach task.

Main Results:

• Successfully retrieved conditional distributions for masked body parts in walking flies.
• Decoded motor trajectories from neural activity and queried the VAE for the encoding distribution in a monkey-reach task.
• Demonstrated the unification of dimensionality reduction and conditional distribution learning within a single VAE framework.

Conclusions:

• The proposed VAE-based approach effectively calculates conditional distributions for neural encoding and decoding.
• This method offers a unified solution for dimensionality reduction and learning complex conditional distributions.
• The approach facilitates the analysis of large-scale, multi-modal neuroscience datasets.