Phase transitions from linear to nonlinear information processing in neural networks.

Area of Science:

• Computational neuroscience
• Machine learning theory
• Complex systems

Background:

• Reservoir computing, particularly echo state networks (ESNs), is a powerful framework for time-series processing.
• Understanding the operational regimes and information processing capabilities of ESNs is crucial for advancing their applications.

Purpose of the Study:

• To investigate the phase transition in information processing from linear to nonlinear regimes within ESNs.
• To characterize the relationship between network nonlinearity, size, and information processing capacity.

Main Methods:

• Systematically varying the standard deviation of input weights to control network nonlinearity.
• Analyzing information processing capacity across different network sizes and noise levels.
• Identifying critical thresholds for the transition to nonlinear processing.

Main Results:

• A critical threshold was identified beyond which information processing capacity increases rapidly.
• The transition becomes sharper with increasing network size, suggesting a discontinuous transition in the infinite-node limit.
• This transition is distinct from the order-to-chaos transition in traditional neural networks.

Conclusions:

• ESNs exhibit a novel phase transition in information processing, moving from linear to nonlinear regimes.
• Network size and noise intensity critically influence this transition and the resulting processing capacity.
• A scaling law was derived, showing critical nonlinearity vanishes without noise, offering insights into ESN design.