Safe Feature Screening for Generalized LASSO | JoVE Visualize

Area of Science:

Machine Learning
Optimization
Statistical Modeling

Background:

Generalized LASSO (GL) problems are computationally intensive, especially with high-dimensional data and complex feature interactions.
Existing feature screening methods are limited to specific interaction structures like chains or trees, hindering broader applicability.
There is a need for robust screening rules that can handle general feature interaction structures in GL problems.

Purpose of the Study:

To develop a novel and efficient safe screening method for Generalized LASSO problems with general feature interaction structures.
To enable the removal or aggregation of inactive features to reduce computational complexity before applying optimization solvers.
To provide a dynamic screening approach that does not require prior knowledge of the solution or regularization parameter.

Main Methods:

Formulated the GL screening problem as a bound estimation task within a large linear inequality system in the dual space.
Proposed a novel bound propagation algorithm for efficient safe screening applicable to general GL problems.
Introduced transformation methods to effectively decouple feature interactions, enhancing the propagation algorithm's performance.

Main Results:

The proposed bound propagation and transformation methods facilitate dynamic screening, allowing screening initiation without pre-existing solution knowledge.
Experimental results on synthetic and real-world datasets validate the effectiveness and efficiency of the developed screening method.
The approach successfully handles general feature interaction structures, overcoming limitations of previous methods.

Conclusions:

The novel bound propagation algorithm offers an efficient and generalizable solution for feature screening in Generalized LASSO problems.
The developed methods significantly reduce the computational burden of GL problems by effectively identifying and removing irrelevant features.
This work advances the field by providing a versatile screening approach applicable to a wider range of complex feature interaction scenarios.