Orthogonal Capsule Networks With Positional Information Preservation and Lightweight Feature Learning

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Traditional deep learning models like transformers and convolutional neural networks (CNNs) often require additional mechanisms to capture positional information.
• Existing methods for object detection can involve complex, multi-stage training processes, separating position and classification tasks.

Purpose of the Study:

• To introduce a novel orthogonal capsule network (OrthogonalCaps) designed for lightweight feature learning that inherently preserves location information.
• To simplify the end-to-end training process for object detection tasks, eliminating the need for separate position regression and object classification.
• To improve the detection of small-scale objects and achieve a better balance between model parameters and accuracy.

Main Methods:

• Developed an orthogonal capsule network (OrthogonalCaps) utilizing orthogonality-based dynamic routing to generate capsule layers.
• Implemented a novel activation function, Capsule ReLU, to mitigate gradient vanishing and enable scale normalization.
• Employed a voting mechanism within the dynamic routing to preserve positional information and reduce parameter count.

Main Results:

• OrthogonalCaps achieved accuracy and run-time performance comparable to Faster R-CNN on the VOC dataset.
• The proposed network demonstrated superior performance in detecting small-scale objects compared to baseline approaches.
• Ablation experiments confirmed the significant contributions of Capsule ReLU and orthogonality-based dynamic routing to classification performance.

Conclusions:

• OrthogonalCaps offers an effective and simplified approach to object detection by integrating location information preservation within a lightweight network architecture.
• The network provides a favorable trade-off between model complexity and detection accuracy, outperforming other capsule network models.
• The proposed methods, Capsule ReLU and orthogonality-based dynamic routing, are crucial for enhancing classification capabilities and adaptability to various object scales.