Testing a single regression coefficient in high dimensional linear models | JoVE Visualize

Area of Science:

Statistics
Econometrics
Machine Learning

Background:

Classical z-tests (or t-tests) for regression coefficients are invalid in high-dimensional data where covariates outnumber samples.
High dimensionality and multicollinearity in linear regression models pose significant challenges for traditional statistical inference.

Purpose of the Study:

To propose a novel and simple method, Correlated Predictors Screening (CPS), for testing the significance of regression coefficients in high-dimensional settings.
To enable the application of z-tests for covariate significance assessment even when predictors are highly correlated.
To achieve consistent model selection through multiple hypothesis testing with controlled false discovery rate.

Main Methods:

Introduction of the Correlated Predictors Screening (CPS) method to manage highly correlated predictors.
Application of ordinary least squares (OLS) for estimating regression coefficients after CPS.
Demonstration of estimator consistency and asymptotic normality, even with heteroscedastic errors.
Multiple hypothesis testing using p-values from covariate significance tests, controlling the false discovery rate (FDR).

Main Results:

The proposed CPS method allows for the valid application of z-tests to assess individual covariate significance.
The resulting estimators are proven to be consistent and asymptotically normal, robust to heteroscedasticity.
Multiple hypothesis testing with controlled FDR leads to consistent model selection.
Simulation studies and empirical examples validate the method's finite sample performance and practical utility.

Conclusions:

The Correlated Predictors Screening (CPS) method provides a statistically sound and practical approach for significance testing in high-dimensional linear regression.
This method effectively addresses challenges posed by multicollinearity, enabling reliable inference and model selection.
The CPS approach enhances the applicability of classical statistical tests in modern, data-rich environments.