A stochastic variational framework for Recurrent Gaussian Processes models.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Dynamical Systems

Background:

• Gaussian Processes (GPs) excel at sequential data but struggle with large datasets.
• Recurrent Gaussian Processes (RGPs) were developed for dynamical data but share scalability issues.
• Existing GP methods are computationally infeasible for very large sequential datasets.

Purpose of the Study:

• To enhance the scalability of Recurrent Gaussian Processes (RGPs) for large-scale sequential data analysis.
• To introduce a novel framework, Stochastic Recurrent Variational Bayes (S-REVARB), for efficient inference in RGPs.
• To enable the application of advanced GP models to massive datasets, particularly in dynamical system identification.

Main Methods:

• Modified the variational approach for RGPs to enable stochastic mini-batch optimization.
• Developed the Stochastic Recurrent Variational Bayes (S-REVARB) framework.
• Proposed Local and Global S-REVARB algorithms to manage computational costs and variational parameters.
• Utilized neural networks as sequential recognition models in the global S-REVARB variant for enhanced scalability.

Main Results:

• The S-REVARB framework significantly improves scalability for GP-based models on large sequential datasets.
• Demonstrated effective dynamical system identification on large-scale datasets, a task challenging for standard RGPs.
• The global S-REVARB variant achieved superior scalability by limiting variational parameter growth.
• Promising results indicate the viability of S-REVARB for massive sequential data analysis.

Conclusions:

• S-REVARB offers a computationally feasible solution for applying powerful hierarchical recurrent GP models to massive sequential data.
• The framework addresses the critical scalability bottleneck in current RGP approaches.
• This work opens new avenues for analyzing large-scale dynamical systems using advanced GP methodologies.