Tucker network: Expressive power and comparison.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) are highly successful in machine learning and computer vision.
• Existing DNN architectures face limitations in expressive power and efficiency.

Purpose of the Study:

• To propose and analyze a novel deep neural network, the Tucker network, based on the Tucker format.
• To investigate the expressive power of the Tucker network compared to shallow networks and hierarchical Tucker (HT) networks.
• To develop a deep version of the Tucker network for enhanced performance.

Main Methods:

• Derivation of the Tucker network from the Tucker tensor format.
• Theoretical analysis of the expressive power of the Tucker network.
• Development of a deep Tucker tensor decomposition by combining hierarchical Tucker and Tucker formats.
• Experimental validation on MNIST, CIFAR-10, and CIFAR-100 datasets.

Main Results:

• The Tucker network exhibits exponentially higher expressive power than shallow networks.
• A shallow network requires exponential width to match the Tucker network's score function.
• The proposed deep Tucker network and Tucker network outperform shallow and HT networks.
• Experimental results validate the theoretical findings on benchmark datasets.

Conclusions:

• The Tucker network offers a significant advancement in expressive power for deep learning models.
• Deep Tucker networks provide a more efficient and performant alternative to existing architectures.
• This work contributes a novel deep tensor decomposition for enhanced machine learning capabilities.