Optimizing Data Flow in Binary Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Binary Neural Networks (BNNs) offer significant inference acceleration through bit-wise operations, replacing floating-point arithmetic.
• Current BNN methods often hinder data flow efficiency due to intermediate bit-width conversions (1 to 16/32 bits).

Purpose of the Study:

• To introduce BNN-Clip, a novel training scheme designed to enhance parallelism and data flow in BNN pipelines.
• To reduce computational overhead and latency in BNNs without compromising accuracy.

Main Methods:

• Implemented a clipping block to reduce data width from 32 to 8 bits within BNN layers.
• Decreased the internal accumulator size of binary layers from 32 bits to mitigate overflow risks without accuracy loss.
• Optimized batch normalization layers for reduced latency and simplified deployment.
• Developed an optimized binary direct convolution implementation for ARM NEON instruction sets.

Main Results:

• Achieved consistent inference latency speed-up, up to 1.3x and 2.4x compared to state-of-the-art BNN frameworks.
• Maintained accuracy comparable to existing state-of-the-art approaches on benchmark datasets (CIFAR-10, SVHN, ImageNet).

Conclusions:

• BNN-Clip effectively improves the efficiency and speed of binary neural networks.
• The proposed methods offer a practical approach to deploying high-performance BNNs on resource-constrained devices.