Active Learning for Enumerating Local Minima Based on Gaussian Process Derivatives.

Area of Science:

• Machine Learning
• Optimization
• Computational Mathematics

Background:

• Identifying local minimum solutions of black-box functions is crucial in many scientific and engineering fields.
• Traditional methods struggle with functions where derivatives are not directly observable.
• Gaussian processes (GPs) offer a probabilistic framework for modeling black-box functions.

Purpose of the Study:

• To develop an efficient active learning (AL) strategy for enumerating all local minimum solutions.
• To address the challenge of unobservable derivatives (gradient and Hessian) in optimization problems.
• To enhance the precision of identifying solutions characterized by zero gradients and positive-definite Hessians.

Main Methods:

• Proposed a novel active learning (AL) algorithm leveraging Gaussian processes (GPs).
• Sequentially selected input points to optimize the update of GP derivative confidence intervals.
• Developed theoretical guarantees for the proposed AL method's convergence and efficiency.

Main Results:

• Demonstrated the effectiveness of the proposed AL method in numerical experiments.
• Showcased improved enumeration of local minimum solutions compared to existing approaches.
• Validated the theoretical analysis through practical simulations.

Conclusions:

• The proposed active learning method efficiently enumerates local minimum solutions for black-box functions.
• This approach effectively handles the challenge of unobservable function derivatives.
• The method provides a robust tool for optimization problems requiring comprehensive local minimum identification.