Sparse Variational Student-t Processes for Heavy-Tailed Modeling

The Gaussian process (GP) is a powerful tool for nonparametric modeling, but its sensitivity to outliers limits its applicability to data distributions with heavy tails. Student-t processes (TPs) offer a robust alternative for heavy-tail modeling, but they lack the scalable developments of the GP to large datasets necessary for practical applications. We present sparse variational TPs (SVTPs), the first principled framework that extends the sparse inducing point method to the TP. We develop two novel inference algorithms, SVTP-UB and SVTP-MC, with theoretical guarantees and derive a natural gradient optimization that exploits a previously unused connection between the Fisher information matrix of the multivariate Student-t distribution and the beta function (the "beta link"). Experiments on UCI and Kaggle datasets demonstrate that SVTP significantly outperforms sparse GPs when the data contain outliers and heavy tails, achieving up to $3\times $ faster convergence and 40% lower prediction error while maintaining computational efficiency for datasets with over 200 000 samples.