Saturating Splines and Feature Selection.

Area of Science:

• Statistics
• Machine Learning
• Computational Mathematics

Background:

• Adaptive regression splines are powerful tools for modeling nonlinear relationships.
• Existing methods often require pre-specified knot locations, limiting flexibility.
• Incorporating saturation, a natural functional property, is crucial for many real-world applications.

Purpose of the Study:

• To extend adaptive regression splines by incorporating saturation.
• To develop an efficient convex optimization algorithm for fitting saturating splines.
• To apply saturating splines within generalized additive models for feature selection and nonlinear fitting.

Main Methods:

• Fitting saturating splines to data using convex optimization over a space of measures.
• Employing an efficient algorithm based on the conditional gradient method.
• Adapting the algorithm for generalized additive models with saturating splines as coordinate functions.

Main Results:

• The proposed algorithm solves the infinite-dimensional optimization problem without pre-specified knot locations.
• The saturation requirement enables simultaneous feature selection and nonlinear function fitting in generalized additive models.
• The method demonstrates effectiveness in capturing bounded functional relationships.

Conclusions:

• Saturating splines offer a flexible and efficient approach to modeling data with inherent saturation.
• The developed optimization framework overcomes limitations of existing spline fitting methods.
• This work provides a foundation for extending the methodology to higher-order splines and diverse boundary conditions.