Convection-Diffusion Equation: A Theoretically Certified Framework for Neural Networks.

Area of Science:

• Computational mathematics
• Machine learning theory
• Artificial intelligence

Background:

• Neural networks exhibit inherent links to network structures, bridging discrete layers with continuous equations.
• Existing research primarily explores ordinary differential equations (ODEs) and feature transformations on input signals.

Purpose of the Study:

• To investigate the partial differential equation (PDE) model of neural networks.
• To establish a theoretical framework connecting neural networks to PDEs, specifically convection-diffusion equations.
• To develop a novel neural network architecture inspired by PDE principles.

Main Methods:

• Viewing neural networks as functionals operating on a base model from the classifier's last layer.
• Applying scale-space theory to derive a convection-diffusion equation for neural network mappings.
• Designing a new network architecture incorporating a diffusion mechanism based on the derived PDE model.

Main Results:

• Theoretical proof that neural network mappings can be formulated by a convection-diffusion equation under specific assumptions.
• Demonstration that this framework encompasses various existing network structures and training techniques.
• Validation of a novel diffusion-based neural network architecture through extensive experiments.

Conclusions:

• The study provides a mathematically grounded PDE framework for understanding neural networks.
• The convection-diffusion equation offers new insights into network behavior and structure.
• The proposed diffusion-based network architecture shows effectiveness on benchmark and real-world datasets.