Ensemble Linear Subspace Analysis of High-Dimensional Data.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional regression analysis faces challenges when the number of covariates (p) exceeds the sample size (n).
• Existing methods often struggle with prediction accuracy in such complex scenarios.

Purpose of the Study:

• To evaluate the performance of ensemble subspace methods in high-dimensional regression.
• To compare the effectiveness of penalty methods, like Lasso, within an ensemble framework.

Main Methods:

• Simulations and a real data example were used to compute mean squared prediction errors.
• Two ensemble Lasso versions were examined: one with cross-validation and one with a trimmed average predictor.
• Linear models with both random and fixed designs were considered.

Main Results:

• The ensemble subspace approach demonstrated improved performance over standard penalty methods.
• Improvements were most notable when covariates had strong associations with the response and model complexity was high.
• The trimmed average version of ensemble Lasso frequently yielded the best predictive performance.

Conclusions:

• Ensemble subspace methods offer a valuable enhancement for high-dimensional regression.
• The trimmed average ensemble Lasso is a robust predictor in complex, high-dimensional settings.
• This approach is particularly beneficial for multivariate mutual information analysis.