Simultaneous Multiple Response Regression and Inverse Covariance Matrix Estimation via Penalized Gaussian Maximum...

Area of Science:

• Statistics
• Machine Learning

Background:

• Multivariate regression is crucial for practical problems, but many methods focus on single responses.
• Current univariate approaches applied separately ignore valuable joint information among multiple response variables.

Purpose of the Study:

• To develop new multivariate regression methods that effectively utilize joint information among response variables.
• To provide sparse estimators for conditional inverse covariance matrices and regression parameters.

Main Methods:

• Proposed three novel methods within a penalized likelihood framework.
• Employed weighted L(1) regularization for sparsity.
• Investigated simultaneous and plug-in estimation strategies for regression parameters and inverse covariance matrices.

Main Results:

• The proposed methods yield sparse estimators for both regression parameters and the conditional inverse covariance matrix.
• Numerical examples show competitive performance in prediction, variable selection, and inverse covariance matrix estimation.
• Asymptotic properties of the developed methods were theoretically explored.

Conclusions:

• The novel multivariate regression methods effectively utilize joint response information.
• These approaches offer advantages over traditional univariate methods for problems with multiple response variables.
• The methods demonstrate strong performance across key statistical and machine learning tasks.