Multivariate multilinear regression

Area of Science:

• Statistics
• Machine Learning
• Computer Vision

Background:

• Conventional regression methods like multivariate linear regression (MLR) and principal component regression (PCR) struggle with high-dimensional data where the number of features exceeds training samples, leading to the under-sample problem (USP).
• The USP in PCR, characterized by a high-dimensional feature space relative to the number of training samples, has received limited attention, impacting regression accuracy and stability.

Purpose of the Study:

• To conduct an in-depth investigation into the under-sample problem (USP) within principal component regression (PCR).
• To propose a novel multivariate multilinear regression (MMR) model as an alternative to MLR for multilinear data.
• To address the principal component selection challenge in PCR and alleviate the USP.

Main Methods:

• Analysis of the causes, conditions, and influence of USP in PCR.
• Development of a multivariate multilinear regression (MMR) model by incorporating the multilinear structure of objects as a constraint on regression coefficients.
• Design of an alternative projection procedure for obtaining regression matrices in MMR due to the absence of a closed-form solution.

Main Results:

• The study elucidates the underlying reasons and conditions for USP in PCR and its impact on regression.
• The proposed MMR model effectively reduces the regression problem to finding low-dimensional coefficients, thereby avoiding principal component selection.
• MMR significantly reduces the necessary sample size, thereby alleviating the USP, and experimental validation on synthetic and AAM fitting data confirms its efficacy.

Conclusions:

• The multivariate multilinear regression (MMR) model offers a viable solution for high-dimensional data with multilinear structures, overcoming limitations of traditional PCR.
• MMR's ability to reduce sample size requirements and avoid principal component selection makes it a powerful tool for addressing the under-sample problem.
• The proposed projection procedure for MMR is computationally analyzed and proven to converge, demonstrating practical applicability, particularly in areas like Active Appearance Model fitting.