Robust deep learning from weakly dependent data

Area of Science:

• Machine Learning
• Statistics
• Deep Learning Theory

Background:

• Existing deep learning theory often assumes bounded loss functions or data.
• This limits applicability to real-world scenarios with unbounded variables and heavy-tailed distributions.

Purpose of the Study:

• To develop robust deep learning theory for weakly dependent observations.
• To establish non-asymptotic bounds for deep neural network estimators with unbounded loss and output.
• To analyze the impact of data moment order (r) on estimator performance.

Main Methods:

• Theoretical analysis of deep neural network estimators under strong mixing and ψ-weak dependence.
• Derivation of non-asymptotic bounds for expected excess risk.
• Investigation of convergence rates based on Hölder smoothness and data moment properties.

Main Results:

• Established non-asymptotic bounds for robust deep learning estimators with finite r-order moments (r>1).
• Demonstrated convergence rates approaching known results when data has moments of any order.
• Showed that rates for exponentially strongly mixing data approach i.i.d. sample rates under specific smoothness conditions.

Conclusions:

• The proposed robust deep learning framework extends theoretical guarantees to unbounded data and loss functions.
• Simulation results confirm the superiority of robust estimators (absolute and Huber loss) over least squares for heavy-tailed errors in regression and autoregression.
• The findings advance the theoretical understanding and practical application of deep learning in robust statistical modeling.