Machine Learning With Tree Tensor Networks, CP Rank Constraints, and Tensor Dropout.

Area of Science:

• Machine Learning
• Computational Physics
• Data Science

Background:

• Tensor networks approximate large tensors using reduced degrees of freedom.
• Canonical polyadic (CP) rank constraints were previously shown to reduce computational costs in quantum many-body physics.

Purpose of the Study:

• To demonstrate the application of tree tensor networks (TTN) with CP rank constraints and tensor dropout in machine learning.
• To evaluate the performance of this approach in image classification tasks.

Main Methods:

• Utilized tree tensor networks (TTN) with imposed CP rank constraints.
• Incorporated tensor dropout as a regularization technique.
• Applied the TTN classifier to the Fashion-MNIST dataset.

Main Results:

• The TTN classifier with CP rank constraints and tensor dropout outperformed other tensor-network-based methods.
• Achieved a test set accuracy of 90.3% on Fashion-MNIST using a low-rank TTN with a branching ratio of 4.
• Demonstrated low computational costs compared to traditional methods.

Conclusions:

• Tensor network classifiers, particularly TTNs with CP rank constraints, offer an effective alternative to deep neural networks.
• CP rank constraints enhance generalization, control overfitting, reduce parameters, and lower computational costs.
• The proposed method effectively addresses the vanishing gradient problem inherent in deep neural networks.