Connections between physics, mathematics, and deep learning.

Area of Science:

• Theoretical Physics
• Machine Learning
• Differential Geometry

Background:

• Classical and quantum mechanics are governed by Fermat's principle of least action.
• The geometry of curved manifolds is described by exterior differential forms.
• Neural network optimization often uses gradient descent, with a loss function guiding the process.

Purpose of the Study:

• To derive neural network equations in an intrinsic, coordinate-invariant manner.
• To explore the connections between deep learning and fundamental physics.
• To provide a formal presentation of the differential geometry of neural networks.

Main Methods:

• Utilizing Fermat's principle of least action from mechanics.
• Applying the theory of exterior differential forms from geometry.
• Formulating neural network equations with the loss function as the Hamiltonian.

Main Results:

• Derived coordinate-invariant neural network equations.
• Introduced a layer metric for pretraining and complex number conjugation.
• Clarified the relationship between gradient descent and Newtonian mechanics.
• Analyzed the Bayesian paradigm as a renormalizable theory, deriving the Bayesian information criterion.

Conclusions:

• The differential formalism offers a new perspective on neural networks, linking them to classical and quantum field theory.
• This approach encourages interdisciplinary research between physics and deep learning.
• Understanding these connections can deepen the appreciation of both fields.