OMNI-CONV: Generalization of the Omnidirectional Distortion-Aware Convolutions

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Omnidirectional images offer vast potential but present challenges due to spherical distortions.
• Conventional convolutional neural networks (CNNs) are designed for perspective images and struggle with omnidirectional data.
• Directly extending CNNs to equirectangular images requires specialized adaptations.

Purpose of the Study:

• To develop a general method for adapting perspective CNNs to equirectangular images.
• To introduce a novel distortion-aware convolution for omnidirectional image processing.
• To demonstrate the effectiveness of the proposed method across various computer vision tasks.

Main Methods:

• Proposed a distortion-aware convolution method to adapt standard CNN architectures for equirectangular images.
• Integrated the novel convolution as a direct replacement for existing convolutional layers.
• Evaluated the adapted models on semantic segmentation, optical flow, and monocular depth estimation tasks.

Main Results:

• The adapted spherical models consistently outperformed their perspective counterparts.
• The method demonstrated generalization across virtual and real outdoor scenarios.
• No additional training cost was required for the adapted convolutional networks.

Conclusions:

• The proposed distortion-aware convolution is an effective and generalizable method for processing omnidirectional images.
• This approach allows leveraging existing CNN architectures for spherical imagery without retraining.
• The technique significantly improves performance on fundamental computer vision tasks using equirectangular images.