Feature Grouping and Selection Over an Undirected Graph.

Area of Science:

• High-dimensional statistics
• Machine learning
• Statistical modeling

Background:

• High-dimensional regression and classification are challenging, particularly with highly correlated features.
• Existing methods like graph-guided fused lasso (GFlasso) leverage feature structure but can introduce estimation bias due to reliance on pairwise sample correlations for feature grouping.

Purpose of the Study:

• To develop advanced feature grouping and selection methods that overcome the limitations of GFlasso.
• To enhance estimation accuracy in high-dimensional settings by refining feature grouping and selection strategies.

Main Methods:

• Proposed three novel methods for feature grouping and selection.
• Method 1: Uses a convex function penalizing the pairwise l_infinity norm for simultaneous grouping and selection.
• Method 2 & 3: Employ non-convex and truncated l_1 regularization, respectively, to further reduce estimation bias. Solved using alternating direction method of multipliers (ADMM) and difference of convex functions (DC) programming.

Main Results:

• The proposed methods effectively combine feature grouping and selection.
• Demonstrated improved estimation accuracy compared to existing approaches.
• Experimental results on synthetic and real datasets validate the effectiveness of the new techniques.

Conclusions:

• The developed methods offer a robust solution for feature selection in high-dimensional, correlated data.
• These approaches mitigate estimation bias, leading to more accurate regression and classification models.
• The study highlights the benefits of advanced regularization techniques for structured feature selection.