Bayesian sparse multiple regression for simultaneous rank reduction and variable selection.

Area of Science:

• Statistics
• Machine Learning
• High-Dimensional Data Analysis

Background:

• High-dimensional regression models often involve matrices with underlying structures like low-rank or row-sparsity.
• Simultaneously estimating these structures is challenging, especially when the number of predictors is large relative to the sample size.

Purpose of the Study:

• To develop a Bayesian methodology for simultaneously estimating low-rank and row-sparse matrices in high-dimensional multiple-response linear regression.
• To provide theoretical support for the methodology, including minimax optimality.
• To propose a practical scheme for variable selection and dimension reduction.

Main Methods:

• A shrinkage prior is devised for the regression coefficient matrix, promoting both low-rank and row-sparse properties without prior rank specification.
• Minimax optimality of the posterior mean is proven under prediction risk in ultra-high dimensional settings.
• A one-step post-processing scheme using group lasso penalties and a novel optimization function for rank estimation are proposed.

Main Results:

• The proposed Bayesian methodology effectively estimates low-rank and row-sparse matrices simultaneously.
• Theoretical analysis confirms the minimax optimality of the posterior mean estimator.
• The post-processing scheme enables efficient variable selection and dimension reduction.

Conclusions:

• The developed Bayesian approach provides a robust framework for analyzing high-dimensional regression with structured coefficient matrices.
• The methodology demonstrates strong theoretical properties and practical utility in variable selection and dimension reduction.
• The approach is validated through simulations and a real-data example.