A two-stage differential evolution algorithm for neural ensemble architecture search

Neural ensembles, comprising multiple heterogeneous neural networks, show promise in machine learning tasks. Their efficacy depends on architecture design, traditionally relying on deep learning expertise. Neural ensemble architecture search methods aim to automate this process, but face challenges in balancing diversity, performance, and computational efficiency. This study introduces DENE, a differential evolution algorithm for NEAS, addressing these limitations. DENE employs a two-stage framework to generate neural ensembles with a multi-head structure, incorporating a surrogate model-based ranking strategy to reduce computational resources during performance evaluation. A novel diversity measurement function enables DENE to optimize for both accuracy and diversity through multi-objective optimization. Experiments on benchmark image classification datasets compare DENE against state-of-the-art evolutionary neural architecture search and NEAS algorithms. Results demonstrate that DENE efficiently generates highly competitive neural ensembles, outperforming existing methods in performance and search time. This research contributes to automated machine learning by providing a more efficient and effective approach to neural ensemble design, potentially broadening the applicability of these powerful models across various domains.