Multiplex visibility graphs to investigate recurrent neural network dynamics.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Complex Systems

Background:

• Recurrent Neural Networks (RNNs) are powerful models for dynamical systems but require careful hyperparameter tuning.
• Echo State Networks (ESNs) are a specific class of RNNs where performance is highly sensitive to hyperparameter settings.
• Current hyperparameter optimization often relies on inefficient trial-and-error methods.

Purpose of the Study:

• To develop principled, unsupervised methods for configuring ESN hyperparameters.
• To enhance ESN performance by minimizing prediction error and maximizing memory capacity.
• To interpret and characterize the internal dynamics of ESNs using a novel graph-based framework.

Main Methods:

• Modeling time series of individual neuron activations using horizontal visibility graphs (HVGs).
• Constructing a multiplex structure where each layer represents the HVG of a neuron's time series.
• Analyzing the topological properties of the multiplex to infer ESN dynamics and guide hyperparameter selection.

Main Results:

• Demonstrated that multiplex topological properties correlate with ESN performance metrics (prediction error, memory capacity).
• Validated the effectiveness of the proposed unsupervised hyperparameter tuning methods on benchmark datasets.
• Showcased the applicability of the approach on a real-world dataset of telephone call records.

Conclusions:

• The proposed graph-based framework provides an effective, unsupervised approach to optimize ESN hyperparameters.
• Analyzing multiplex horizontal visibility graphs offers valuable insights into ESN internal dynamics.
• This method offers a principled alternative to trial-and-error for improving ESN performance in various applications.