Nonlinear variable selection with continuous outcome: a fully nonparametric incremental forward stagewise approach.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Sparse generalized additive models (GAMs) are powerful tools for analyzing complex datasets.
• Effective variable selection is crucial for building accurate and interpretable GAMs, especially with high-dimensional data.
• Existing methods often rely on pre-specified functional forms, limiting their applicability.

Purpose of the Study:

• To develop a novel variable selection method for sparse generalized additive models.
• To create a method that does not assume any specific functional form for predictors.
• To enhance model interpretability and performance by efficiently selecting relevant variables from a large candidate pool.

Main Methods:

• Introduced an incremental forward stagewise regression approach for variable selection.
• Developed a novel 'roughening' technique to adjust residuals during iterative model fitting without functional form assumptions.
• The method is implemented within the R package 'nlnet' available on CRAN.

Main Results:

• Simulation studies demonstrate the new method's competitiveness against popular machine learning approaches.
• The method's performance was validated using diverse real-world datasets.
• The approach successfully identifies relevant variables without prior knowledge of their functional relationships.

Conclusions:

• The proposed variable selection method offers a flexible and effective alternative for sparse generalized additive models.
• The 'roughening' technique addresses the challenge of unknown functional forms in GAM variable selection.
• The nlnet package provides accessible implementation for researchers and practitioners.