EvolCAF: Automatic Cost-Aware Acquisition Function Design Using Large Language Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Optimization

Background:

• Cost-aware Bayesian optimization (BO) is crucial for problems with expensive, variable evaluation costs.
• Designing efficient cost-aware acquisition functions (AFs) is a major challenge in BO.
• Manual design of AFs requires significant expertise and effort.

Purpose of the Study:

• To introduce EvolCAF, a novel framework for automatic design of cost-aware acquisition functions (AFs).
• To leverage large language models (LLMs) and evolutionary computation (EC) for AF design.
• To reduce reliance on domain expertise and manual trial-and-error in AF development.

Main Methods:

• EvolCAF integrates LLMs with EC to automatically design cost-aware AFs.
• The framework utilizes evolutionary concepts like crossover and mutation in the algorithmic space.
• It analyzes historical data, surrogate models, and budget information for AF design.

Main Results:

• The best AF designed by EvolCAF effectively utilizes available information (data, models, budget).
• EvolCAF-designed AFs introduce novel concepts not seen in prior research.
• The approach demonstrates remarkable efficiency and generalization across diverse tasks compared to EIpu and EI-cool.

Conclusions:

• EvolCAF offers a new paradigm for designing cost-aware acquisition functions.
• The framework automates AF design, reducing manual effort and domain expertise requirements.
• EvolCAF-designed AFs show superior performance and interpretability in cost-aware Bayesian optimization.