Efficient-CapsNet: capsule network with self-attention routing

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) are inefficient for large datasets due to redundant feature detectors.
• Capsule networks (CapsNets) offer a promising solution for efficient encoding of feature affine transformations and generalization to novel viewpoints.
• Current CapsNet research has not fully explored their parameter efficiency.

Purpose of the Study:

• To investigate the efficiency of capsule networks in terms of parameter count and generalization.
• To develop and evaluate an extreme capsule network architecture with significantly reduced parameters.
• To introduce a novel, non-iterative routing algorithm for capsule networks.

Main Methods:

• Designed an extreme capsule network architecture with approximately 160 K parameters.
• Replaced the traditional dynamic routing with a novel, highly parallelizable, non-iterative routing algorithm.
• Evaluated the proposed architecture on three different datasets, comparing performance against existing methods.

Main Results:

• The proposed extreme capsule network achieved state-of-the-art results using only 2% of the parameters of the original CapsNet.
• The novel routing algorithm effectively handles a reduced number of capsules.
• The methodology proved effective across various capsule network implementations.

Conclusions:

• Capsule networks can achieve high performance with drastically reduced parameters, demonstrating significant efficiency gains.
• The novel routing algorithm enhances the parallelizability and efficiency of capsule networks.
• Capsule networks show strong potential for embedding visual representations that generalize well to new viewpoints.