Materials science optimization benchmark dataset for multi-objective, multi-fidelity optimization of hard-sphere...

Area of Science:

• Computational science
• Materials science
• Optimization

Background:

• Effective scientific benchmarks are crucial for progress, but often lack real-world task resemblance or have high computational costs.
• Existing benchmarks may impede progress due to inadequate difficulty or relevance, and lack accessibility.
• Creating surrogate models that are indistinguishable from ground truth observations is a key objective.

Purpose of the Study:

• To develop a benchmark dataset for chemistry and materials science optimization tasks that closely mimics real-world complexity.
• To create a surrogate model that accurately represents complex simulation data, including noise and failure regions.
• To ensure the benchmark has low computational overhead for accessibility and repeatability.

Main Methods:

• Performed 494,498 hard-sphere packing simulations (206 CPU days) with nine input parameters, linear constraints, and two discrete fidelities.
• Logged simulation data into a MongoDB Atlas database, generating failure probability and regression datasets.
• Developed a surrogate model using percentile ranks to account for heteroskedastic noise, avoiding conventional a-priori assumptions.

Main Results:

• Generated two core tabular datasets: failure probability and regression, mapping input parameters to outcomes.
• The surrogate model successfully incorporated simulation failure and heteroskedastic noise, closely approximating actual simulation results.
• The percentile rank method provided reliable and accurate data, differing from traditional noise modeling approaches.

Conclusions:

• The developed surrogate model and data handling techniques effectively bridge the gap between low-overhead benchmarks and complex real-world optimization scenarios.
• This approach enhances the relevance and accessibility of scientific benchmarks in fields like chemistry and materials science.
• The methodology is extendable to other benchmark datasets, promoting more realistic and computationally efficient scientific discovery.