Convolution in Convolution for Network in Network.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Network in Network (NiN) enhances deep convolutional neural networks by incorporating multilayer perceptrons (MLP) for non-linear feature extraction.
• NiN's use of dense MLP layers improves feature representation but leads to a high parameter count.

Purpose of the Study:

• To address the parameter inefficiency of NiN by proposing a sparse MLP approach.
• To introduce a novel method, Convolution in Convolution (CiC), for reducing parameters in NiN architectures.

Main Methods:

• Replacing dense shallow MLP layers with sparsely connected MLP layers.
• Implementing sparsity in the channel dimension or channel-spatial domain.
• Utilizing unshared convolutions across channels and shared convolutions across spatial dimensions within computational layers.

Main Results:

• The proposed Convolution in Convolution (CiC) method effectively reduces parameters in NiN models.
• Experimental results on CIFAR10, augmented CIFAR10, and CIFAR100 datasets demonstrate the effectiveness of CiC.
• The sparse MLP approach maintains or improves recognition performance compared to dense MLP.

Conclusions:

• Convolution in Convolution (CiC) offers an efficient alternative to dense MLPs in Network in Network architectures.
• The proposed method successfully balances feature representation power with parameter reduction.
• CiC presents a viable strategy for developing more efficient deep learning models.