Evolving Fully Automated Machine Learning via Life-Long Knowledge Anchors.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Automated machine learning (AutoML) has advanced significantly, yet existing pipelines often exclude crucial steps like data cleaning and model ensembling.
• This partial automation leads to substantial human effort and less-than-optimal results, particularly across diverse data types (image, text, tabular).

Purpose of the Study:

• To develop a comprehensive, fully automated machine learning (AutoML) pipeline that integrates all stages from data preprocessing to model ensembling.
• To address the challenges of vast search spaces and generalization across different data modalities in AutoML.

Main Methods:

• Introduction of a novel "life-long" knowledge anchor design to accelerate search within the extensive AutoML pipeline.
• Integration of knowledge anchors with an evolutionary algorithm for joint optimization of all pipeline components.
• Comprehensive automation of data preprocessing, feature engineering, model selection, training, and ensembling.

Main Results:

• The proposed fully AutoML pipeline achieved state-of-the-art performance on multiple datasets and modalities.
• The framework secured the sole championship in the NeurIPS 2019 AutoDL challenge, outperforming other methods across image, video, speech, text, and tabular data tracks.

Conclusions:

• The developed fully AutoML pipeline demonstrates superior performance and efficiency by automating the entire machine learning workflow.
• This approach significantly reduces human involvement and enhances generalization capabilities for diverse machine learning tasks.