Sparse Reduced Rank Huber Regression in High Dimensions

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional data analysis presents challenges due to noise and complexity.
• Existing reduced rank regression methods often overlook heavy-tailed noise characteristics.
• Robust statistical methods are crucial for reliable analysis of complex datasets.

Purpose of the Study:

• To develop a robust regression method for high-dimensional data with heavy-tailed noise.
• To establish theoretical guarantees for the proposed method's estimation accuracy.
• To analyze the trade-off between noise properties and statistical bias.

Main Methods:

• Proposing a sparse reduced rank Huber regression.
• Employing convex relaxation of a non-convex optimization problem.
• Utilizing block coordinate descent and alternating direction method of multipliers algorithms.

Main Results:

• Established non-asymptotic estimation error bounds under Frobenius and nuclear norms.
• Quantified the trade-off between noise heavy-tailedness and statistical bias.
• Demonstrated convergence rates dependent on noise moment bounds, matching sub-Gaussian rates for second-moment bounded noise.

Conclusions:

• The proposed sparse reduced rank Huber regression effectively handles high-dimensional data with heavy-tailed noise.
• Theoretical analysis provides crucial insights into the method's performance under varying noise conditions.
• Numerical studies and a data application validate the method's practical utility.