Balanced Decoupled Spatial Convolution for CNNs

In this paper, we are interested in designing lightweight CNNs by decoupling the convolution along the spatial and channel dimension. Most existing decoupling techniques focus on approximating the filter matrix through decomposition. In contrast, we provide a decoupled view of the standard convolution to separate the spatial information and the channel information. The resulting decoupled process is exactly equivalent to the standard convolution. Inspired from our decoupled view, we propose an effective structure, balanced decoupled spatial convolution (BDSC), to relax the sparsity of the filter in spatial aggregation by learning a spatial configuration and reduce the redundancy by reducing the number of intermediate channels. We also designed an adaptive spatial configuration, which is simply adding a nonlinear activation layer [rectified linear units (ReLU)] after the intermediate output. Our experiments verify that the adaptive spatial configuration can improve the classification performance without extra cost. In addition, our BDSC achieves comparable classification performance with the standard convolution but with a smaller model size on Canadian Institute for Advanced Research (CIFAR)-100, CIFAR-10, and ImageNet. To show the potential of further reducing the redundancy of across channel-domain convolution, we also show experiments of our models with a designed lightweight across channel-domain convolution. Finally, we show in our experiments that our models achieve superior performance than the state-of-the-art models.