Statistical agnostic regression: A machine learning method to validate regression models.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Regression analysis is fundamental for modeling relationships between variables.
• Linear regression, including Ordinary Least Squares (OLS), Ridge, and Lasso, is widely used.
• Machine learning (ML) models are increasingly applied but often lack formal statistical significance testing.

Purpose of the Study:

• Introduce Statistical Agnostic Regression (SAR) for evaluating ML-based linear regression.
• Develop a method to formally assess the statistical significance of linear relationships.
• Provide a robust framework for hypothesis testing in regression analysis.

Main Methods:

• Utilize concentration inequalities to analyze expected loss under worst-case scenarios.
• Define a threshold for determining statistical significance with a specified probability (1-η).
• Employ a non-parametric approach, avoiding assumptions of classical regression methods.

Main Results:

• The SAR test demonstrates comparable analysis of variance to the classical multivariate F-test.
• SAR effectively controls the false positive rate, unlike standard ML methods.
• Residuals from SAR offer a balance between ML and OLS approaches.

Conclusions:

• SAR offers a statistically rigorous and assumption-free method for linear regression significance testing.
• The proposed method enhances the reliability of ML-based regression models.
• SAR provides a valuable tool for researchers seeking robust statistical inference.