Deep learning in random neural fields: Numerical experiments via neural tangent kernel.

Area of Science:

• Computational neuroscience
• Machine learning theory
• Deep learning architectures

Background:

• Biological neural networks in the cortex exhibit properties like receptive fields and correlated connection weights.
• Understanding supervised learning in these complex neural fields is crucial for advancing AI.
• Recent machine learning theory (neural tangent kernel regime) suggests global minima exist in over-parameterized random networks.

Purpose of the Study:

• To investigate supervised learning in a multilayer neural field model.
• To compare the performance of this neural field model against randomly connected deep networks.
• To analyze the robustness and generalization capabilities of the proposed neural field architecture.

Main Methods:

• Developed a multilayer neural field model with continuously distributed receptive fields.
• Incorporated initial connection weights that are random but spatially correlated within layers.
• Empirically compared the model's performance against standard deep networks using supervised learning tasks.
• Investigated model behavior through the lens of the neural tangent kernel regime.

Main Results:

• The proposed neural field model exhibits robustness to input patterns deformed by noise.
• The generalization ability of the multilayer neural field model is slightly superior to conventional models.
• Numerical analysis confirmed that global minima exist within small neighborhoods for these neural fields, aligning with neural tangent kernel theory.

Conclusions:

• The structured, correlated nature of neural fields offers advantages over purely random connections in deep networks.
• The developed neural field model provides a promising direction for more robust and generalizable AI systems.
• Further research into biologically inspired neural field architectures can yield significant advancements in machine learning.