Benchmarking AutoML for regression tasks on small tabular data in materials design

Area of Science:

• Materials Science
• Computer Science
• Data Science

Background:

• Machine learning (ML) is increasingly vital in materials engineering.
• Automated machine learning (AutoML) adoption is rising globally for data analysis.
• Existing AutoML benchmarks often overlook the specific challenges of small, experimental materials datasets (<1000 samples).

Purpose of the Study:

• To benchmark AutoML frameworks specifically for materials engineering with small tabular datasets.
• To compare AutoML performance against manual data analysis under these conditions.
• To assess the performance, robustness, and usability of AutoML in materials science.

Main Methods:

• Evaluation of four representative AutoML frameworks.
• Testing across twelve diverse, domain-specific materials engineering datasets.
• Focus on performance, robustness, and usability metrics.

Main Results:

• AutoML demonstrates high competitiveness compared to manual model optimization, even with limited training time.
• The choice of data sampling strategy for training and testing significantly impacts result reliability.
• AutoML tools offer practical value for materials engineers working with limited experimental data.

Conclusions:

• AutoML is a viable and competitive alternative to manual methods in materials engineering, particularly for small datasets.
• Effective data sampling strategies are paramount for achieving dependable outcomes with AutoML in materials science.
• This study provides essential guidance for adopting AutoML in experimental materials research.