Bayesian Optimization Based on K-Optimality.

Area of Science:

• Statistics
• Machine Learning
• Experimental Design

Background:

• Gaussian process (GP) surrogate models in Bayesian optimization (BO) face challenges with unstable predictions due to ill-conditioned Gram matrices.
• Determining the optimal balance between exploration and exploitation in BO is difficult.

Purpose of the Study:

• To address GP prediction instability and the exploration-exploitation trade-off in Bayesian optimization using K-optimality.
• To develop novel methods for robust design of experiments (DoE) and efficient optimization.

Main Methods:

• Introduced K-optimality, focusing on minimizing the condition number of the kernel's Gram matrix.
• Proposed Sequentially Bayesian K-optimal design (SBKO) using K-optimality as an acquisition function for stable GP predictions.
• Developed K-optimal enhanced Bayesian Optimization (KO-BO), specifically K-optimal enhanced Expected Improvement (KO-EI), to automatically determine exploration-exploitation parameters.

Main Results:

• SBKO demonstrated superior performance over Monte Carlo, Latin hypercube sampling, and sequential DoE by maximizing posterior variance with high prediction precision.
• KO-EI exhibited a higher convergence rate and smaller variance compared to the classical Expected Improvement (EI) method in optimization problems.
• Both SBKO and KO-BO effectively reduce integrated posterior variance and maximize hyper-parameter information gain.

Conclusions:

• K-optimality provides a robust framework for enhancing Gaussian process predictions and optimizing Bayesian optimization strategies.
• The proposed SBKO and KO-BO methods offer significant improvements in prediction stability, efficiency, and performance for design of experiments and optimization tasks.