Cellular automata as convolutional neural networks.

Area of Science:

• Artificial Intelligence
• Computational Science
• Dynamical Systems Theory

Background:

• Deep learning excels at predicting complex dynamical systems.
• Understanding how neural networks encode dynamical rules is crucial.
• Cellular automata (CA) are discrete dynamical systems challenging to analyze with traditional methods.

Purpose of the Study:

• To investigate how neural networks represent dynamical rules of cellular automata.
• To develop a general neural network architecture for learning CA dynamics.
• To explore the relationship between CA rule complexity and neural network representations.

Main Methods:

• Representing cellular automata using convolutional neural networks (CNNs) with network-in-network architectures.
• Developing a general convolutional multilayer perceptron (MLP) architecture.
• Training networks on video data of CA to learn their dynamical rules.
• Employing information-theoretic techniques to analyze layer activation patterns and network structure.

Main Results:

• A general convolutional MLP architecture can learn arbitrary CA rules from video data.
• Training dynamics show high similarity across replicates for large network widths.
• Common structural patterns emerge in networks trained on different CA rulesets.
• Network representation complexity (hierarchical vs. shallow) correlates with CA rule complexity.

Conclusions:

• Neural networks can effectively learn and represent discrete dynamical systems like CA.
• The internal structure of trained networks reflects the complexity of the learned CA rules.
• This work provides insights into how neural network entropy relates to the representation of physical processes.