Consistent group selection in high-dimensional linear regression

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Group Lasso is effective for variable selection when covariates have natural groupings.
• High-dimensional settings, where the number of groups exceeds sample size, pose challenges for standard methods.
• Understanding the selection and estimation properties of Group Lasso in these settings is crucial.

Purpose of the Study:

• To analyze the selection and estimation performance of the Group Lasso in high-dimensional scenarios.
• To develop an improved method, the adaptive Group Lasso, for more accurate variable selection.
• To establish conditions for the consistency of the adaptive Group Lasso.

Main Methods:

• Investigated the theoretical properties of the Group Lasso in high-dimensional settings.
• Proposed an adaptive Group Lasso method, generalizing the adaptive Lasso.
• Analyzed the consistency of the adaptive Group Lasso using the Group Lasso as an initial estimator.

Main Results:

• The Group Lasso can achieve model dimension comparable to the true model and is estimation consistent under certain conditions.
• The Group Lasso is generally not selection consistent and may select irrelevant groups.
• The proposed adaptive Group Lasso demonstrates consistency in group selection under specific conditions.

Conclusions:

• While Group Lasso offers benefits in structured high-dimensional data, its selection consistency is limited.
• The adaptive Group Lasso provides a more reliable approach for variable selection in such scenarios.
• The adaptive Group Lasso's performance is contingent on the quality of the initial estimator, ideally Group Lasso.