Neural networks with optimized single-neuron adaptation uncover biologically plausible regularization

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Dynamical systems theory

Background:

• Biological neurons exhibit adaptive input-output properties like heterogeneous f-I curves and spike frequency adaptation.
• These cellular properties are thought to optimize neural coding under changing stimuli.
• Understanding how neural circuits leverage single-neuron flexibility remains a challenge.

Purpose of the Study:

• To investigate how single-neuron adaptive mechanisms can be exploited by brain circuits.
• To explore how network-level requirements shape cellular function using computational models.
• To systematically study single-neuron input-output adaptation in an end-to-end optimized artificial neural network.

Main Methods:

• Utilizing artificial neural networks (ANNs) with adaptable nonlinear activation functions.
• Parametrizing neuron activation functions to mimic biological f-I curves.
• Implementing both static individual neuron adaptation and real-time shared adaptation mechanisms.
• Applying dynamical systems theory to analyze emergent network properties.

Main Results:

• Adaptive ANNs demonstrated significantly improved robustness against noise and input statistical variations.
• Emergent single-neuron properties in optimized networks were found to play an active regularization role.
• These networks showed enhanced capabilities in optimally propagating information over time.

Conclusions:

• Single-neuron adaptability is crucial for robust and efficient information processing in neural circuits.
• Artificial neural networks can effectively model and uncover principles of biological neural computation.
• Optimized adaptive mechanisms in ANNs mirror biological coding strategies like gain scaling.