Discontinuities in recurrent neural networks.

Area of Science:

• Computational theory
• Artificial intelligence
• Neural networks

Background:

• Classical analog recurrent neural networks (ARNNs) use polynomial net functions and continuous activation functions.
• The computational power of ARNNs is limited by precision requirements linear to computation time.

Purpose of the Study:

• To investigate the computational power of discontinuous real computational models based on ARNNs.
• To introduce and analyze arithmetic networks, an extension of ARNNs.
• To compare the capabilities of arithmetic networks with existing computational models.

Main Methods:

• Augmenting ARNNs with discontinuous neurons (e.g., threshold, zero test) to create arithmetic networks.
• Analyzing computational complexity with polynomial time computable real weights.
• Comparing arithmetic networks to the Blum-Shub-Smale model.
• Investigating the impact of adding periodic functions.

Main Results:

• Arithmetic networks with polynomial time computable real weights can compute arbitrarily complex recursive functions.
• These networks are equivalent to the Blum-Shub-Smale model with bounded registers.
• Arithmetic networks with rational weights require exponential precision for simulation.
• Arithmetic networks with real weights do not have fixed precision bounds, unlike ARNNs.
• Incorporating periodic functions makes these networks computationally equivalent to massively parallel machines.

Conclusions:

• Arithmetic networks possess significantly greater computational power than classical ARNNs.
• Discontinuous networks can solve PSPACE-complete problems efficiently.
• These findings have implications for understanding the limits of computation in neural networks.