A framework for measuring the training efficiency of a neural architecture

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Measuring efficiency in neural network system development is an open research problem.
• Existing methods may not fully capture the nuances of training efficiency across different architectures and tasks.

Purpose of the Study:

• To present an experimental framework for measuring the training efficiency of neural network architectures.
• To analyze and compare the training efficiency of Convolutional Neural Networks (CNNs) and Bayesian Convolutional Neural Networks (BCNNs).

Main Methods:

• Developed an experimental framework to quantify neural network training efficiency.
• Evaluated the framework using CNNs and BCNNs on MNIST and CIFAR-10 datasets.
• Analyzed the impact of training progression, stopping criteria, and model size on efficiency.

Main Results:

• Training efficiency decays over time and varies with stopping criteria and model size.
• CNNs demonstrated greater training efficiency compared to BCNNs on both datasets.
• The complexity of the learning task amplifies the efficiency differences between architectures.

Conclusions:

• The proposed framework provides a robust method for assessing neural network training efficiency.
• Overtraining can confound efficiency measurements, highlighting the importance of appropriate stopping criteria.
• Architectural choices significantly impact training efficiency, especially for complex tasks.