SplitWise regression for capturing nonlinear effects in interpretable model selection.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Regression modeling often struggles to balance capturing nonlinear relationships with maintaining model interpretability.
• Classical linear models offer transparency but cannot effectively model complex nonlinear patterns.
• Existing interpretable nonlinear methods may lack flexibility or introduce complexity.

Purpose of the Study:

• Introduce SplitWise, a novel stepwise regression framework designed to bridge the gap between linear interpretability and nonlinear flexibility.
• To develop a method that adaptively transforms numeric predictors into binary features only when beneficial for model fit.
• To provide a regression approach that remains interpretable and verifiable while capturing threshold-based nonlinear effects.

Main Methods:

• SplitWise employs shallow decision trees to identify optimal thresholds for transforming numeric predictors into binary features.
• Model fit is assessed using Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC) to guide predictor transformation.
• The framework integrates these binary features into a globally linear equation, preserving overall model transparency.

Main Results:

• On synthetic data, SplitWise reduced median Root Mean Squared Error (RMSE) by 7-14% compared to linear baselines.
• Variable selection accuracy was improved, with Matthews Correlation Coefficient (MCC) up to ~0.79 versus ~0.51 for LASSO.
• On real datasets (Wine Quality, Bodyfat), SplitWise achieved comparable or improved RMSE with fewer selected predictors than other methods.

Conclusions:

• SplitWise offers an interpretable yet flexible approach to regression modeling, effectively capturing threshold-based nonlinearities.
• The framework enhances predictive accuracy and variable selection compared to traditional linear models and some advanced methods.
• SplitWise represents a valuable tool for researchers and practitioners seeking transparent models with enhanced predictive power.