Dynamic memory-enhanced recurrent neural networks with temporal attention for robust long-range connectivity...

Area of Science:

• Computational Neuroscience
• Machine Learning
• Signal Processing

Background:

• Accurate estimation of long-range directed connectivity is crucial for understanding neural networks but is challenging due to vanishing gradients.
• Conventional recurrent neural networks struggle with modeling long-range dependencies and preserving temporal information.

Purpose of the Study:

• To propose a novel dynamic memory-enhanced LSTM architecture with a temporal attention mechanism (DM+Attention) for improved gradient-based connectivity inference.
• To enhance the modeling of long-range dependencies and the accuracy of connectivity estimation in time-series data.

Main Methods:

• Introduced a trainable dynamic memory matrix to improve long-range dependency modeling.
• Integrated a temporal attention mechanism to refine gradient estimates by focusing on relevant time steps.
• Compared DM+Attention with bidirectional LSTM, Transformer encoder, and multilayer perceptron using synthetic time-series data.

Main Results:

• DM+Attention achieved a mean AUC of 0.959, significantly outperforming bLSTM (25.7%), Transformer encoder (16.0%), and partial directed coherence (19.2%).
• Performance gains were more pronounced under dense connectivity conditions, with DM+Attention achieving a mean AUC of 0.907.
• Achieved high correlation coefficients (R²=0.986 sparse, R²=0.691 dense) in continuous connectivity-weight prediction.

Conclusions:

• The combination of dynamic memory and temporal attention effectively captures complex temporal dependencies for enhanced connectivity inference.
• The proposed framework is scalable, biologically plausible, and applicable to neuroimaging data (EEG, MEG, fMRI).
• Introduced an efficient non-parametric statistical approach for distinguishing genuine connectivity from noise.