Shapley value optimized differentiable architecture search for lightweight neural networks in resource-constrained...

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Efficient neural networks are crucial for resource-constrained environments like embedded systems and IoT devices.
• Differentiable Architecture Search (DARTS) optimizes architectures but struggles with large search spaces and operation importance.
• Existing methods often lack efficiency and fail to adequately represent the contribution of different operations.

Purpose of the Study:

• To propose a novel Shapley value-based method for optimizing the search space in Differentiable Architecture Search (DARTS).
• To fairly measure the contribution of each operation type during architecture search.
• To develop lightweight neural networks with enhanced efficiency for resource-limited applications.

Main Methods:

• Implemented a Shapley value-based approach to fairly assess the contribution of each operation type within the DARTS search space.
• Integrated operation types with their corresponding connection nodes to reduce redundancy.
• Compressed the neural network search space, leading to more efficient architectures.

Main Results:

• Achieved competitive accuracy on CIFAR-10 (95.45%) with only 2.7M parameters and 0.26 GPU days of search cost.
• Attained 76.14% accuracy on ImageNet with 4.3M parameters and 12.2 GPU days of search cost.
• Demonstrated superior performance and efficiency compared to existing Neural Architecture Search (NAS) methods.

Conclusions:

• The proposed method significantly enhances the efficiency of neural architecture search.
• It effectively reduces computational overhead and topology complexity in neural networks.
• This approach offers a superior alternative to existing NAS methods, particularly for resource-constrained environments.