Sparse Variational Student-t Processes for Heavy-Tailed Modeling

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Gaussian processes (GPs) are effective for nonparametric modeling but struggle with outlier-sensitive data.
• Student-t processes (TPs) provide robustness for heavy-tailed distributions but lack scalability for large datasets.

Purpose of the Study:

• To introduce sparse variational Student-t processes (SVTPs), a scalable framework for robust heavy-tail modeling.
• To develop novel inference algorithms and optimization techniques for SVTPs.

Main Methods:

• Extended sparse inducing point methods to Student-t processes.
• Developed two inference algorithms: SVTP-UB and SVTP-MC.
• Derived a natural gradient optimization using the beta link connection.

Main Results:

• SVTPs demonstrate superior performance over sparse GPs on datasets with outliers and heavy tails.
• Achieved up to 3x faster convergence and 40% lower prediction error.
• Maintained computational efficiency for datasets exceeding 200,000 samples.

Conclusions:

• SVTPs provide a scalable and robust solution for nonparametric modeling with heavy-tailed data.
• The developed methods offer significant improvements in prediction accuracy and convergence speed compared to existing sparse GP methods.