Learning Channel-Wise Interactions for Binary Convolutional Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Conventional binary convolutional neural networks (BCNNs) suffer from significant information loss due to quantization errors and inconsistent pixel signs in binary feature maps.
• This loss hinders the efficiency and accuracy of BCNNs in tasks requiring precise inference.

Purpose of the Study:

• To propose novel binary convolutional neural network approaches that mitigate information loss and improve inference efficiency.
• To address the sign inconsistency problem in binary feature maps inherent in traditional BCNNs.

Main Methods:

• Developed channel-wise interaction based binary convolutional neural networks (CI-BCNN) utilizing reinforcement learning to mine channel-wise interactions and impose priors.
• Introduced hierarchical channel-wise interaction based binary convolutional neural networks (HCI-BCNN) to optimize the search space via hierarchical reinforcement learning.
• Proposed a denoising interacted bitcount operation to alleviate noise in channel-wise priors by smoothing interactions.

Main Results:

• CI-BCNN and HCI-BCNN effectively alleviate sign inconsistencies in binary feature maps, preserving crucial information during inference.
• Experimental results on CIFAR-10 and ImageNet datasets validate the superior performance of the proposed CI-BCNN and HCI-BCNN methods.
• The novel approaches demonstrate enhanced efficiency and accuracy compared to conventional BCNNs.

Conclusions:

• The proposed CI-BCNN and HCI-BCNN methods represent a significant advancement in binary convolutional neural networks, offering improved efficiency and accuracy.
• Exploiting channel-wise interactions and employing denoising techniques are effective strategies for overcoming limitations in BCNNs.
• These findings pave the way for more robust and efficient deep learning models in resource-constrained environments.