Fisher duty interval and particle swarm optimization for neural architecture search

Automatically constructing Deep Neural Networks (DNNs) has become a key focus in artificial intelligence research, as their performance is highly dependent on the architecture and parameters of the network. Careful selection of the architecture and parameters for specific tasks can significantly improve the network's output. This study proposes an intelligent and efficient framework for Neural Architecture Search (NAS) by integrating Particle Swarm Optimization (PSO), the concept of Fisher Duty Interval (FDI), Transfer Learning (TL), and graph-based architecture representation. Given the high dependency of DNN performance on architecture design, narrowing the search space and guiding it toward promising regions is essential for improving model effectiveness. In the initial phase, the Fisher Information Matrix (FIM) is used to compute the statistical distance FDI between the target task and a set of previously solved tasks. This serves as the basis for TL, allowing the generation of informed initial particles from previously successful architectures. PSO then performs a multi-level search, balancing global and local exploration. In early iterations, candidate architectures are evaluated relative to baseline architectures, while in later stages, comparison shifts dynamically toward the global best particle. Architectures are encoded as Directed Acyclic Graphs (DAGs) with cell-based modules, allowing for heterogeneous and flexible designs where each cell can have a distinct structure. Additionally, weight inheritance and FIM-based performance estimation reduce the need for full training during each iteration, minimizing computational cost. Overall, this framework leverages prior knowledge, reduces search complexity, and efficiently explores the architecture space to discover models that outperform baseline designs and random search results. The effectiveness of this approach has been tested through classification tasks on well-known datasets, with comprehensive results reported accordingly.