Effect of recurrent infomax on the information processing capability of input-driven recurrent neural networks

Reservoir computing is a framework for exploiting the inherent transient dynamics of recurrent neural networks (RNNs) as a computational resource. On the basis of this framework, much research has been conducted to evaluate the relationship between the dynamics of RNNs and the RNNs' information processing capability. In this study, we present a detailed analysis of the information processing capability of an RNN optimized by recurrent infomax (RI), an unsupervised learning method that maximizes the mutual information of RNNs by adjusting the connection weights of the network. The results indicate that RI leads to the emergence of a delay-line structure and that the network optimized by the RI possesses a superior short-term memory, which is the ability to store the temporal information of the input stream in its transient dynamics.