Efficient sparse modeling with automatic feature grouping.

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Mining

Background:

• High-dimensional data analysis often requires grouping similar features to enhance model stability and interpretability.
• Existing sparse-modeling approaches like OSCAR facilitate feature grouping but suffer from high computational costs.
• Efficient feature selection and grouping are crucial for improving generalization in machine learning models.

Purpose of the Study:

• To develop a computationally efficient solver for the Octagonal Shrinkage and Clustering Algorithm for Regression (OSCAR).
• To reduce the time complexity of OSCAR's optimization procedure for high-dimensional datasets.
• To maintain or improve the feature grouping capabilities of OSCAR while enhancing its practical applicability.

Main Methods:

• Proposed an efficient solver for OSCAR utilizing an accelerated gradient method.
• Developed a simple iterative group merging algorithm to solve the key proximal step of the OSCAR optimization.
• Analyzed the theoretical time complexity reduction from O(d^2) to O(d) for d input features.

Main Results:

• The new solver dramatically reduces the empirical time complexity of OSCAR from O(d^2) to O(d).
• Experimental results validate the efficiency and effectiveness of the proposed solver on various datasets.
• The enhanced OSCAR maintains its competitive performance in sparse modeling and automatic feature grouping.

Conclusions:

• The proposed accelerated gradient-based solver significantly improves the computational efficiency of OSCAR.
• This advancement makes OSCAR a more practical and scalable tool for feature grouping in high-dimensional data.
• The efficient OSCAR solver facilitates better data interpretation and improved model generalization through automatic feature grouping.