Dynamic memory-enhanced recurrent neural networks with temporal attention for robust long-range connectivity...

Accurate estimation of long-range directed connectivity remains a critical challenge in recurrent neural network design due to vanishing gradients over long sequences. In this work, we propose a dynamic memory-enhanced LSTM architecture augmented with a temporal attention mechanism (DM + Attention) to address the limitations of conventional recurrent neural networks in gradient-based connectivity inference. Specifically, we introduce a trainable dynamic memory matrix that improves the modeling of long-range dependencies while preserving critical temporal information. The temporal attention mechanism further refines gradient estimates by selectively focusing on the most relevant time steps. We use extensive synthetic time-series data to compare our DM and its attention-enhanced variant (DM + Attention) models with several established methods, including bidirectional LSTM, Transformer encoder, and multilayer perceptron. Across all evaluated connectivity densities, DM + Attention achieved a mean AUC of 0.959, improving over bLSTM, Transformer encoder, and partial directed coherence by 25.7%, 16.0%, and 19.2%, respectively. This advantage was larger under dense connectivity conditions, where DM + Attention achieved a mean AUC of 0.907 and improved over bLSTM and Transformer encoder by 40.4% and 32.9%, respectively. In continuous connectivity-weight prediction, DM + Attention achieved a correlation coefficient between the predicted and target weights (R² = 0.986) in the sparse condition and maintained (R² = 0.691) in the densest condition. We also present a computationally efficient non-parametric statistical approach that appends random noise variables to distinguish genuine connectivity from background noise, thus obviating the need for extensive repetitions of surrogate-based tests. Overall, our findings underscore the effectiveness of combining dynamic memory mechanisms and temporal attention to capture complex temporal dependencies and enhance the interpretability of gradient-based connectivity estimates. This framework is both scalable and biologically plausible for neural connectivity inference and can be readily extended to real-world neuroimaging applications such as EEG, MEG, or fMRI.