Effective Automatic Method Selection for Nonlinear Regression Modeling.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Statistical Modeling

Background:

• Automatic method selection is crucial for nonlinear regression.
• Standard nonlinear least squares estimators can be sensitive to outliers.
• Theoretical recommendations for estimator selection in nonlinear regression are limited.

Purpose of the Study:

• To explore metalearning for automatic selection of robust estimators in nonlinear regression.
• To propose and evaluate novel metalearning approaches for this task.
• To identify the most effective estimator for nonlinear regression with potential outliers.

Main Methods:

• Four metalearning approaches for automatic method selection were proposed.
• Extensive computations were performed on a database of 643 real-world datasets.
• A novel approach combined supervised feature selection (Random Forest) and oversampling (SMOTE).

Main Results:

• The proposed metalearning approaches demonstrated effectiveness in selecting appropriate estimators.
• The combination of Random Forest and SMOTE significantly improved results, mitigating issues with imbalanced data.
• The nonlinear least weighted squares estimator was found to outperform other estimators in a substantial percentage of datasets.

Conclusions:

• Metalearning provides a viable and effective alternative to theoretical approaches for nonlinear regression estimator selection.
• The hybrid method utilizing Random Forest and SMOTE is a promising strategy for robust method selection.
• The nonlinear least weighted squares estimator shows strong performance and robustness, particularly in the presence of data outliers.