On the computational power of Elman-style recurrent networks

Recently, Elman (1991) has proposed a simple recurrent network which is able to identify and classify temporal patterns. Despite the fact that Elman networks have been used extensively in many different fields, their theoretical capabilities have not been completely defined. Research in the 1960's showed that for every finite state machine there exists a recurrent artificial neural network which approximates it to an arbitrary degree of precision. This paper extends that result to architectures meeting the constraints of Elman networks, thus proving that their computational power is as great as that of finite state machines.