Learning multisensory integration and coordinate transformation via density estimation.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Brain sensory processing involves multisensory integration, coordinate transformations, and prior information incorporation.
• Optimal sensory processing requires maintaining correct posterior distributions, with optimality observed in behavioral studies.
• The complexity and plasticity of sensory modalities suggest these computations are learned, but the learning mechanism remains unclear.

Purpose of the Study:

• To provide a principled computational model for how the brain learns complex sensory processing operations.
• To treat the acquisition of sensory mappings as a density estimation problem using machine learning.
• To demonstrate a unified approach for learning various neural computations in sensory processing.

Main Methods:

• Modeled sensory mapping acquisition as density estimation, a machine learning and statistics problem.
• Used unisensory-population activities as observed data, synaptic connections as fixed parameters, and multisensory-population activities as latent variables.
• Trained a restricted Boltzmann machine with the contrastive-divergence rule.

Main Results:

• Successfully learned multiple neural computations, including optimal integration, prior encoding, hierarchical integration, conditional integration, and coordinate transformation.
• Demonstrated a biologically plausible learning rule (contrastive-divergence) for complex sensory processing.
• The model provides a unified framework for understanding diverse sensory computations.

Conclusions:

• The brain's learning of sensory processing can be effectively modeled using density estimation principles from machine learning.
• Restricted Boltzmann machines with contrastive-divergence offer a viable computational framework for neural learning.
• The model generates testable predictions regarding multisensory representations in the brain.