Bayesian³ Active Learning for the Gaussian Process Emulator Using Information Theory

Area of Science:

• Machine Learning
• Computational Science
• Bayesian Inference

Background:

• Gaussian Process Emulators (GPEs) approximate computationally intensive models.
• Effective training set selection is crucial for GPEs in Bayesian inference.
• Existing methods lack optimal strategies for adaptive GPE training.

Purpose of the Study:

• To present a fully Bayesian framework for GPEs integrated with Bayesian Active Learning (BAL).
• To introduce and evaluate three novel BAL strategies for GPE training set selection.
• To compare the performance of these strategies in terms of efficiency and accuracy.

Main Methods:

• Developed three BAL strategies based on information-theoretic principles: Bayesian model evidence, relative entropy, and information entropy.
• Applied these strategies to analytical and carbon-dioxide benchmark problems.
• Assessed GPE performance through convergence analysis and post-calibration uncertainty quantification.

Main Results:

• All three BAL strategies demonstrated convergence towards reference solutions.
• Bayesian model evidence and relative entropy strategies outperformed the information entropy strategy.
• The relative entropy-based strategy exhibited superior performance compared to the Bayesian model evidence strategy.

Conclusions:

• Bayesian active learning significantly enhances the efficiency of Gaussian Process Emulators.
• Relative entropy is the most effective BAL strategy for GPE training, offering superior performance and reliable uncertainty quantification.
• Information entropy-based BAL can be misleading in this context.