On the Identifiability and Interpretability of Gaussian Process Models.

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Gaussian processes (GPs) are powerful tools for probabilistic modeling.
• Matérn kernels are widely used in GP models due to their flexibility in defining function smoothness.
• Additive mixtures of Matérn kernels are common in single-output GPs, but their properties are not fully understood.

Purpose of the Study:

• To critically examine the properties of additive mixtures of Matérn kernels in single-output GP models.
• To explore the potential of multiplicative mixtures of Matérn kernels for multi-output GP models.
• To analyze the identifiability of parameters in both additive and multiplicative kernel mixtures.

Main Methods:

• Theoretical derivation of kernel properties for additive and multiplicative mixtures.
• Analysis of parameter identifiability in single- and multi-output GP models.
• Extensive simulations and real-world applications to validate findings.

Main Results:

• For additive mixtures, the smoothness is dictated by the least smooth component, rendering individual component parameters unidentifiable.
• For multiplicative mixtures in multi-output GPs, the covariance matrix is identifiable up to a multiplicative constant.
• These findings hold for both single- and multi-output GP settings.

Conclusions:

• Additive mixtures of Matérn kernels in single-output GPs provide limited flexibility and identifiability.
• Multiplicative mixtures offer a promising approach for multi-output GPs, enhancing model interpretability.
• The study underscores the importance of selecting appropriate kernel structures for specific GP modeling tasks.