Architecture Knowledge Distillation for Evolutionary Generative Adversarial Network.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Generative Adversarial Networks (GANs) excel at image generation but suffer from unstable training, limiting their practical use.
• One-shot neural architecture search (NAS) for GANs often results in poorly optimized subnetworks due to inherited weights, further degrading performance.

Purpose of the Study:

• To address the instability and performance degradation issues in GAN training and NAS.
• To propose a novel framework, Architecture Knowledge Distillation for Evolutionary GAN (AKD-EGAN), for improved GAN architecture search and training.

Main Methods:

• Employs a two-stage approach: Architecture Knowledge Distillation (AKD) during supernet training to optimize subnetworks and accelerate learning.
• Utilizes a multi-objective evolutionary algorithm (MOEA) for efficient searching of optimal subnet architectures based on multiple performance metrics.
• Incorporates a strategy for effective architecture inheritance to enhance GAN stability and image quality.

Main Results:

• AKD-EGAN demonstrates superior performance compared to state-of-the-art methods in GAN image generation.
• Achieved a Fréchet Inception Distance (FID) of 7.91 and an Inception Score (IS) of 8.97 on the CIFAR-10 dataset.
• Obtained competitive results on the STL-10 dataset with an FID of 20.32 and an IS of 10.06.

Conclusions:

• AKD-EGAN effectively improves GAN training stability and image generation quality.
• The proposed method offers an efficient and effective solution for neural architecture search in GANs.
• Code and models are publicly available for further research and application.