A methodology for the design of experiments in computational intelligence with multiple regression models.

Area of Science:

• Computational Intelligence
• Machine Learning
• Statistical Modeling

Background:

• Experimental design and result validation are crucial in research.
• Machine Learning (ML) regression techniques in Computational Intelligence (CI) are complex and require robust comparison methods.
• The RRegrs R package offers ten regression models for predictive modeling.

Purpose of the Study:

• To propose and validate a novel statistical framework for comparing ML regression models in CI.
• To evaluate the significance and relevance of model selection using the proposed methodology.
• To provide a comprehensive, modifiable methodology for comparing results in CI and related fields.

Main Methods:

• Implementation of a statistical framework for experimental design and validation.
• Evaluation against the established RRegrs R package using five simple and three complex real-world datasets.
• Application of statistical significance testing to compare model performance.

Main Results:

• The proposed framework yielded different best models compared to RRegrs on three out of five simple datasets.
• Model selection using the new methodology was found to be statistically significant and relevant.
• Different best models were identified for three complex datasets compared to the previously published methodology.

Conclusions:

• The developed statistical methodology offers a significant improvement for comparing ML regression models in CI.
• The framework's validation demonstrates its effectiveness and statistical relevance for model selection.
• The proposed approach provides a flexible and open methodology applicable to various scientific fields.