Bayesian Multi-objective Hyperparameter Optimization for Accurate, Fast, and Efficient Neural Network Accelerator...

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Resource-constrained environments necessitate efficient intelligent systems on edge devices.
• Neuromorphic hardware offers a pathway for embedding intelligence with low power consumption.
• Hyperparameter optimization is critical for designing effective neural networks on this hardware.

Purpose of the Study:

• To develop a multi-objective Bayesian hyperparameter optimization framework for neuromorphic systems.
• To maximize neural network performance while minimizing energy and area requirements.
• To address hyperparameter challenges across input/output encoding, network design, and hardware.

Main Methods:

• Implemented a hierarchical pseudo agent-based multi-objective Bayesian optimization framework.
• Validated the approach on digital and mixed-signal (memristor-based) neural accelerators.
• Tested on control and classification applications, including Pole-Balance and RoboNav.

Main Results:

• The framework effectively optimizes hyperparameters for performance, energy, and area.
• Significant performance improvements were observed for specific applications (e.g., Pole-Balance: 52-71%).
• Demonstrated the framework's ability to show varying hyperparameter impact across different applications and system modules.

Conclusions:

• The proposed framework is effective for optimizing neuromorphic hardware design in resource-limited settings.
• Hyperparameter optimization significantly impacts application performance, with varying degrees of influence.
• The study highlights the importance of considering system-wide hyperparameters for efficient neuromorphic computing.