Evolutionary bi-level neural architecture search with training: A framework for color classification

The design of Artificial Neural Networks (ANNs) for classification tasks has been a topic of interest. However, defining an optimal ANN architecture remains challenging, especially when considering resource constraints and the large number of design parameters. This paper proposes an Evolutionary Bi-Level Neural Architecture Search with Training (EB-LNAST) approach that simultaneously optimizes the architecture, weights, and biases of a neural network using a bi-level optimization strategy. The upper level focuses on minimizing the network complexity penalized by the lower level performance function, while the lower level optimizes training parameters to minimize the loss function and maximize the predictive performance. The proposal is evaluated on a real-world color classification task and the WDBC dataset, demonstrating statistically significant improvements over traditional machine learning algorithms, as well as advanced models. Compared to Multilayer Perceptron (MLP) based algorithms, EB-LNAST achieves superior predictive performance when the architecture is fixed, and remains competitive, with a marginal reduction in performance of no more than [Formula: see text], even when compared against MLPs optimized with extensive hyperparameter tuning, including architecture, activation functions, regularization, and optimizers. Remarkably, EB-LNAST achieves up to a [Formula: see text] reduction in model size, highlighting its ability to discover compact and efficient architectures. EB-LNAST is a reliable alternative for generating compact and effective neural network architectures in accordance with the problem's requirements, enabling efficient exploration of the search space while maintaining or exceeding the predictive performance of state-of-the-art classification algorithms.