Supervised Learning for Nonsequential Data: A Canonical Polyadic Decomposition Approach.

Area of Science:

• Machine Learning
• Computational Statistics
• Data Science

Background:

• Supervised learning for nonsequential tasks requires modeling feature interactions.
• High-order feature interactions lead to the curse of dimensionality.
• Existing tensor network (TN) methods like tensor train (TT) are sensitive to feature ordering.

Purpose of the Study:

• To develop a feature ordering invariant method for modeling feature interactions.
• To improve the efficiency and performance of supervised learning models for nonsequential data.
• To generalize tensor-based models for enhanced expressiveness.

Main Methods:

• Representing model parameters as a weight tensor.
• Utilizing canonical polyadic decomposition (CPD) for feature ordering invariance.
• Developing inference and learning algorithms with regularization and initialization.
• Generalizing the framework with feature mapping to high-dimensional vectors and normalization.

Main Results:

• The proposed CP-based predictor outperforms other TN-based predictors on sparse data.
• Comparable performance to other TNs on dense nonsequential tasks.
• Significant performance improvements on dense tasks using feature mapping and normalization, matching fully connected neural networks.

Conclusions:

• CPD provides a robust and efficient alternative to other TNs for nonsequential tasks.
• The generalized framework enhances model expressiveness and performance, particularly on dense data.
• This approach offers a promising direction for advanced feature interaction modeling.