A two-stage differential evolution algorithm for neural ensemble architecture search.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Neural ensembles, combinations of diverse neural networks, offer potential for machine learning tasks.
• Designing effective neural ensemble architectures traditionally requires significant deep learning expertise.
• Automated neural ensemble architecture search (NEAS) methods struggle to balance model diversity, performance, and computational cost.

Purpose of the Study:

• To introduce DENE, a differential evolution algorithm designed for efficient and effective neural ensemble architecture search (NEAS).
• To address the limitations of existing NEAS methods in balancing ensemble diversity, performance, and computational efficiency.

Main Methods:

• DENE utilizes a two-stage framework to generate neural ensembles with a multi-head structure.
• A surrogate model-based ranking strategy is employed to minimize computational resource usage during performance evaluation.
• A novel diversity measurement function facilitates multi-objective optimization for both accuracy and diversity.

Main Results:

• DENE demonstrated efficient generation of highly competitive neural ensembles across benchmark image classification datasets.
• The proposed algorithm outperformed state-of-the-art evolutionary neural architecture search and NEAS methods.
• DENE achieved superior performance and reduced search time compared to existing approaches.

Conclusions:

• DENE provides a more efficient and effective method for designing neural ensembles, advancing automated machine learning.
• This research enhances the automated design of neural ensembles, potentially expanding their application across diverse domains.