An Optimization-Based Method for Feature Ranking in Nonlinear Regression Problems.

Area of Science:

• Machine Learning
• Data Science
• Optimization

Background:

• Feature ranking is crucial for decision support systems and dimensionality reduction.
• Real-world data often requires methods to assess feature relevance effectively.
• Existing methods may not fully capture the nuances of feature importance in regression tasks.

Purpose of the Study:

• To develop a new feature ranking method for regression problems.
• To formally define feature relevance using a neural network inversion problem.
• To provide an effective alternative to existing feature ranking techniques.

Main Methods:

• Formulated feature relevance as a minimum zero-norm inversion problem of a neural network.
• Employed a concave approximation for the non-smooth zero-norm function.
• Defined and solved a smooth, global optimization problem for feature relevance assessment.

Main Results:

• The proposed feature ranking method demonstrates effectiveness comparable to existing approaches.
• Computational experiments on artificial and real datasets validate the method's performance.
• The method's computational cost (CPU time) can be a limitation for large-dimensional problems.

Conclusions:

• The new feature ranking method offers a valid alternative for assessing feature relevance in regression.
• The approach effectively handles the complexity of feature importance in continuous function approximations.
• Further research may be needed to optimize computational efficiency for large-scale applications.