High-dimensional feature selection by feature-wise kernelized Lasso.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Supervised feature selection aims to identify predictive input features.
• The standard Lasso method efficiently selects features based on linear dependencies.
• Capturing nonlinear input-output relationships remains a challenge for traditional Lasso.

Purpose of the Study:

• To develop a kernelized Lasso method for feature selection that captures nonlinear dependencies.
• To enable efficient computation of globally optimal solutions for high-dimensional data.
• To demonstrate the method's effectiveness in classification and regression tasks.

Main Methods:

• A feature-wise kernelized Lasso approach is proposed.
• Kernel functions are utilized to capture nonlinear input-output dependencies.
• Kernel-based independence measures, like the Hilbert-Schmidt independence criterion, are employed.

Main Results:

• The method successfully identifies nonredundant features with strong statistical dependence.
• Efficient computation of globally optimal solutions is achieved, ensuring scalability.
• Experiments show effectiveness on datasets with thousands of features for classification and regression.

Conclusions:

• Kernelized Lasso offers a scalable and effective solution for feature selection in the presence of nonlinear dependencies.
• The approach enhances the capabilities of Lasso for complex, high-dimensional datasets.
• This method advances feature selection techniques in machine learning and statistical modeling.