Variational tensor neural networks for deep learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Physics

Background:

• Deep neural networks (NNs) face scalability challenges with increasing neuron counts, limiting network depth.
• Existing NN architectures struggle with vast parameter spaces, hindering performance on complex tasks.

Purpose of the Study:

• To develop a scalable neural network architecture overcoming depth and parameter limitations.
• To introduce a novel tensor neural network (TNN) by integrating tensor networks (TN) into NN frameworks.
• To enable efficient training of deep learning models with a large number of parameters.

Main Methods:

• Integration of tensor networks (TN) into neural network (NN) architectures.
• Development of a variational DMRG-inspired training technique for TNNs.
• Utilizing a local gradient-descent approach for tensor gradient computation, allowing hybrid dense and tensor layers.

Main Results:

• Demonstrated a scalable tensor neural network (TNN) architecture.
• Achieved efficient training over a large parameter space.
• Provided benchmark results for regression, classification, and image recognition (MNIST) validating TNN accuracy and efficiency.

Conclusions:

• The proposed TNN architecture effectively addresses scalability limitations in deep learning.
• The variational training algorithm enables efficient handling of large parameter spaces and provides insights into model entanglement.
• TNNs offer a promising direction for developing deeper and more efficient neural networks.