Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Individual neurons exhibit complex and diverse intrinsic dynamics.
• Heterogeneity in neuronal dynamics may enhance network computation for temporally complex data.

Purpose of the Study:

• To investigate the role of complex and heterogeneous neuronal dynamics in network computation.
• To develop and evaluate a novel rate-based neuronal model, the generalized-leaky-integrate-and-fire-rate (GLIFR) model.

Main Methods:

• Developed the GLIFR model, a differentiable rate-based model incorporating multiple dynamical mechanisms, including after-spike currents.
• Employed machine learning techniques, specifically gradient descent, to optimize synaptic weights and intrinsic neuronal parameters.
• Trained GLIFR networks on temporally challenging tasks, such as sequential MNIST.

Main Results:

• GLIFR networks learned diverse parameters, leading to heterogeneity in neuronal dynamics.
• Networks demonstrated robustness to random neuronal silencing.
• GLIFR networks showed mixed performance compared to standard recurrent neural networks, excelling in pixel-by-pixel tasks but underperforming in line-by-line tasks.

Conclusions:

• Neuronal complexity and diversity contribute to computational robustness in neural networks.
• The GLIFR model offers a feasible method for training complex neuronal networks using exact gradients.
• After-spike currents and learned heterogeneity are key factors in performance gains.