Predicting Machine Learning Pipeline Runtimes in the Context of Automated Machine Learning

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Automated machine learning (AutoML) aims to optimize machine learning pipelines for maximum prediction performance.
• A significant challenge in AutoML is the inefficient use of computational resources, often leading to pipeline evaluations being canceled due to timeouts.
• Evaluating numerous candidate pipelines is computationally expensive and can be unproductive if they fail to complete within allocated time.

Purpose of the Study:

• To develop an approach for predicting the runtime of two-step machine learning pipelines, including those with a pre-processor.
• To enable automated systems to anticipate and avoid pipeline timeouts, thereby improving the efficiency of the AutoML process.
• To enhance the successful evaluation of machine learning pipelines within computational constraints.

Main Methods:

• Developing runtime models for individual algorithms used in machine learning pipelines.
• Training separate offline runtime models for each potential algorithm.
• Deriving an overall runtime prediction from the individual algorithm models for a complete pipeline.

Main Results:

• The proposed approach effectively predicts the runtime of machine learning pipelines.
• It leads to an increase in the number of successfully completed pipeline evaluations in an AutoML tool.
• The method preserves or enhances the quality of the best solutions found by the AutoML tool.

Conclusions:

• Runtime prediction is a viable strategy to mitigate timeouts in AutoML.
• This approach optimizes computational resource utilization in automated machine learning.
• The method contributes to more effective and efficient model development through AutoML.