HexNet: An Orientation-Aware Deep Learning Framework for Omni-Directional Input.

Area of Science:

• Computer Vision
• Deep Learning
• Spherical Data Processing

Background:

• Traditional deep learning methods struggle with spherical data, introducing distortions and discontinuities.
• Existing spherical convolutional neural networks (CNNs) are computationally expensive, limiting their use to low resolutions and shallow architectures.

Purpose of the Study:

• To develop an efficient and orientation-aware deep learning framework for processing spherical signals.
• To enable high-resolution analysis of spherical data without distortions or excessive memory usage.

Main Methods:

• Proposed HexNet, an orientation-aware deep learning framework utilizing efficient sphere projection for standard planar network operations.
• Introduced a graph-based HexNet version for partial spheres, enabling high-resolution analysis with residual network architectures.
• Developed kernels operating on the sphere's tangent, allowing transfer of pre-trained feature weights from perspective data.

Main Results:

• HexNet achieved state-of-the-art performance in semantic segmentation on the 2D3DS and omni-directional SYNTHIA datasets.
• Demonstrated reduced distortion effects compared to planar CNNs on the Cityscapes dataset.
• Successfully implemented object detection for the spherical domain and presented rotation-invariant classification and segmentation.

Conclusions:

• HexNet offers a distortion-free and computationally efficient solution for deep learning on spherical data.
• The framework enables high-resolution analysis and facilitates pretraining on large datasets like ImageNet.
• HexNet establishes a new state of the art for various tasks in spherical domain processing.