Efficient learning and feature selection in high-dimensional regression.

Area of Science:

• Machine Learning
• Statistical Modeling
• High-Dimensional Data Analysis

Background:

• High-dimensional regression presents challenges in efficient learning and feature selection.
• Standard regression models often lack robustness and automatic feature relevance detection.
• Existing methods may struggle with real-time incremental learning requirements.

Purpose of the Study:

• To introduce a novel algorithm for efficient learning and feature selection in high-dimensional regression.
• To develop a probabilistic and statistically robust approach to generalized linear regression.
• To enhance existing methods with automatic relevance detection and computational advantages.

Main Methods:

• Modification of standard regression models to derive a probabilistic backfitting version.
• Utilizing the expectation-maximization algorithm and variational approximation methods.
• Developing a variational Bayesian least squares (VBLS) approach for generalized linear regression.

Main Results:

• The VBLS approach provides a statistically robust, black-box method for high-dimensional regression.
• It enables automatic relevance detection of input features.
• VBLS forms the core of sparse Bayesian learning, offering computational and robustness benefits over methods like the relevance vector machine.

Conclusions:

• The VBLS algorithm is a suitable drop-in replacement for generalized linear regression techniques.
• Its iterative nature supports real-time incremental learning, crucial for robotics and neuroprosthetics.
• The method demonstrates suitability on synthetic, neurophysiological, and benchmark datasets.