General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification.

Area of Science:

• Machine Learning
• Computational Mathematics
• Data Science

Background:

• Model expressive power in machine learning often scales with size, posing challenges for large-scale applications.
• Low-rank tensor methods offer an efficient approach to manage high-dimensional data in machine learning.
• Existing tensor learning methods typically focus on specific tasks, lacking a unified framework.

Purpose of the Study:

• To propose a generic Bayesian framework for a broad range of tensor learning problems.
• To develop a low-rank tensor prior for automatic rank determination in nonlinear models.
• To enable uncertainty quantification in tensor learning outcomes.

Main Methods:

• Developed a generic Bayesian framework for tensor learning.
• Introduced a low-rank tensor prior for automatic rank determination.
• Implemented the framework using stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD).

Main Results:

• The proposed framework successfully determines tensor rank automatically.
• The method effectively quantifies the uncertainty of the obtained results.
• Demonstrated performance on tensor completion and tensorized neural network training tasks.

Conclusions:

• The generic Bayesian framework provides a unified approach to tensor learning.
• Automatic rank determination and uncertainty quantification are key advantages.
• The framework is applicable to diverse tensor learning problems, enhancing model robustness and interpretability.