The Geometry of Enhancement in Multiple Regression

Area of Science:

• Statistics
• Regression Analysis

Background:

• Linear multiple regression models are widely used for prediction and inference.
• The concept of 'enhancement' describes a specific scenario in regression analysis where model performance exceeds expectations based on individual predictor correlations.

Purpose of the Study:

• To define and geometrically explain the phenomenon of enhancement in linear multiple regression.
• To derive equations for understanding the conditions under which enhancement occurs and is maximized.
• To provide computational tools for simulating regression models exhibiting enhancement.

Main Methods:

• Mathematical derivation using standardized regression coefficients and predictor-criterion correlations.
• Geometrical interpretation in p-dimensional space using hyper-ellipsoids.
• Eigen decomposition of the predictor correlation matrix.
• Development of R code for simulation.

Main Results:

• Enhancement is defined by the inequality R^2 > r'r, where R^2 is the model's R-squared and r'r is the sum of squared correlations between predictors and the criterion.
• Enhancement is impossible for simple linear regression (p=1).
• Maximum enhancement occurs when the vector of standardized regression coefficients (b) is collinear with the eigenvector corresponding to the smallest eigenvalue (λp) of the predictor correlation matrix (Rxx).
• The sets of vectors leading to enhancement (B1 and B2) are characterized and their dimensionality explored.

Conclusions:

• The study provides a comprehensive mathematical and geometrical framework for understanding enhancement in multiple regression.
• The findings clarify the conditions for enhancement and identify the specific predictor-criterion relationships that maximize it.
• The provided R code enables empirical investigation and simulation of enhancement phenomena.