Informational properties of neural nets performing algorithmic and logical tasks

Area of Science:

• Computational neuroscience
• Theoretical computer science
• Information theory

Background:

• Biological neural networks exhibit complex computational capabilities.
• Algorithmic approaches, like Turing machines, provide a framework for understanding computation.
• The relationship between genetic encoding and neural processing remains an area of active research.

Purpose of the Study:

• To propose that genetic information for algorithmic neural processors is encoded by the entropy of minimal Turing machines.
• To construct a near-minimal Turing machine capable of bivalent propositional logic.
• To demonstrate that neural networks can compute propositional logic and be trained via error back-propagation.

Main Methods:

• Construction of a near-minimal Turing machine for n-variable bivalent propositional logic.
• Development of neural network models to perform the same logical tasks.
• Utilizing informational entropy to compare neural nets and Turing machines.
• Applying error back-propagation for training neural networks.

Main Results:

• The informational entropy of analogous Turing machines represents the genetic information for algorithmic neural processors.
• Neural networks were shown to compute propositional logic.
• A direct correlation between the entropy of neural nets and their analogous Turing machines was established.
• Single-hidden-layer neural networks successfully learned logic algorithms through error back-propagation.

Conclusions:

• Genetic information for tutoring biological neural nets can be understood through Turing machine entropy.
• Neural networks are capable of implementing complex logical algorithms.
• Error back-propagation is an effective training method for neural networks performing logical computations.