Fast and accurate Bayesian optimization with pre-trained transformers for constrained engineering problems.

Area of Science:

• Engineering Design Optimization
• Machine Learning
• Artificial Intelligence

Background:

• Bayesian Optimization (BO) is crucial for complex engineering design with expensive black-box functions.
• Existing methods often struggle with multiple constraints, requiring separate models for each.

Purpose of the Study:

• Introduce a novel constraint-handling framework for BO using Prior-data Fitted Networks (PFNs).
• Enable simultaneous evaluation of objectives and constraints within a single model pass.

Main Methods:

• Leveraged PFNs, a foundation transformer model, for integrated objective and constraint handling.
• Employed in-context learning for efficient evaluation.
• Benchmarked across 15 diverse engineering design problems.

Main Results:

• Achieved an order of magnitude speedup compared to traditional Gaussian Process (GP)-based methods.
• Maintained or improved solution quality.
• Demonstrated particular effectiveness in rapidly finding feasible, optimal solutions for engineering problems.

Conclusions:

• The PFN-based BO framework offers a significant advancement in efficient engineering design optimization.
• This approach accelerates the discovery of feasible and optimal solutions in complex design spaces.