Adaptive Huber Regression on Markov-dependent Data.

Area of Science:

• Statistics
• Time Series Analysis
• Machine Learning

Background:

• High-dimensional linear regression commonly assumes independent data and sub-Gaussian errors, which are often violated in real-world time-series.
• Existing methods like Adaptive Huber Regression (AHR) address heavy-tailed errors but typically assume data independence.
• Real-world high-dimensional time-series data frequently exhibit dependent structures and heavy-tailed error distributions.

Purpose of the Study:

• To extend Adaptive Huber Regression (AHR) to accommodate dependent observations, particularly those with a Markov dependence structure.
• To theoretically justify the performance of AHR in the context of dependent high-dimensional time-series data.
• To analyze how Markov dependence influences the adaptation of the robustification parameter and the estimation accuracy of regression coefficients.

Main Methods:

• Developed a theoretical framework to analyze Adaptive Huber Regression (AHR) under Markov dependence.
• Investigated the impact of the spectral gap of the Markov chain on the estimation properties.
• Derived theoretical guarantees for the robustification parameter adaptation and coefficient estimation.

Main Results:

• Demonstrated that Markov dependence significantly affects the optimal robustification parameter in AHR.
• Showed that the effective sample size for estimation is discounted by a factor related to the Markov chain's spectral gap.
• Established theoretical bounds for the estimation error of regression coefficients under Markov dependence.

Conclusions:

• Adaptive Huber Regression (AHR) can be effectively applied to high-dimensional time-series data with Markov dependence.
• The spectral gap of the Markov chain is a crucial factor in determining the performance and sample size requirements for AHR.
• This work provides theoretical foundations for robust regression in dependent, heavy-tailed time-series settings.