Semi-supervised linear regression: enhancing efficiency and robustness in high dimensions.

Area of Science:

• Machine Learning
• Statistics
• Statistical Learning Theory

Background:

• Current understanding in semi-supervised learning posits that unlabeled data benefits linear parameter estimation solely under model misspecification.
• This paradigm is challenged in high-dimensional statistical settings where unlabeled data can offer advantages even for correctly specified models.

Purpose of the Study:

• To challenge the prevailing understanding of unlabeled data utility in semi-supervised learning.
• To demonstrate the benefits of incorporating unlabeled samples in high-dimensional settings for linear parameter estimation.
• To develop robust and efficient semi-supervised estimators for regression coefficients.

Main Methods:

• Development of robust semi-supervised estimators for regression coefficients, initially focusing on dense scenarios without assuming sparse population slopes.
• Extension of methods for enhanced efficiency in sparse linear slope scenarios.
• Extensive numerical studies to validate the performance of the proposed semi-supervised methods.

Main Results:

• Demonstrated that additional unlabeled samples improve estimation accuracy for linear parameters in high-dimensional settings, contrary to prior beliefs.
• Showcased that leveraging unlabeled data reduces estimation bias and enhances inference robustness, even when the true model is linear.
• Proposed novel semi-supervised methods offering improved efficiency, particularly in sparse linear slope scenarios.

Conclusions:

• Unlabeled data provides significant benefits for parameter estimation in semi-supervised learning within high-dimensional contexts, irrespective of model specification.
• The developed robust semi-supervised estimators effectively reduce bias and improve accuracy and robustness in regression analysis.
• The proposed methods offer practical advancements for utilizing unlabeled data in statistical modeling.