Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed...

Area of Science:

• Machine Learning
• Computational Statistics
• Dynamic Systems Modeling

Background:

• Scalable Gaussian Processes (GPs) are crucial for large datasets.
• Karhunen-Loève (KL) decomposition offers a promising, inducing-point-free approach to GP scalability.
• High dimensionality resulting from KL decomposition necessitates effective variable selection.

Purpose of the Study:

• To develop a novel forward variable selection method for KL-decomposed GPs.
• To enable efficient and accurate modeling of dynamic systems.
• To reduce computational complexity in GP training and inference.

Main Methods:

• Utilized the ordered basis functions of the Bayesian Smoothing Spline ANOVA (BSS-ANOVA) kernel's KL expansion.
• Implemented fast Gibbs sampling within a fully Bayesian framework.
• Applied the method to dynamic systems identification by modeling tangent space dynamics.

Main Results:

• Achieved competitive accuracy and reduced training/inference times on tabular datasets.
• Demonstrated effectiveness on 'Susceptible, Infected, Recovered' (SIR) and 'Cascaded Tanks' datasets.
• Showcased suitability for dynamic systems identification tasks.

Conclusions:

• The proposed variable selection method effectively limits terms in KL-expanded GPs.
• The approach offers a computationally efficient and accurate solution for dynamic systems identification.
• The method shows promise compared to Random Forests, ResNets, and SINDy for specific tasks.