Revisiting 360 Depth Estimation With PanoGabor: A New Fusion Perspective

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Monocular 360 depth estimation is crucial for 3D environment perception.
• Inherent distortions and wide field of view (FoV) in 360 images present significant challenges.
• Existing methods often rely on intermediate representations (e.g., Cubemap) that reintroduce distortions when converted to equirectangular projection (ERP).

Purpose of the Study:

• To propose an oriented-distortion-aware Gabor Fusion framework (PGFuse) for monocular 360 depth estimation.
• To address the challenges posed by distortions and large FoV in 360 images.
• To enhance feature extraction and depth cue utilization.

Main Methods:

• Introduction of Gabor filters for frequency-domain texture analysis, expanding receptive fields and enhancing depth cues.
• Development of a latitude-aware distortion representation to create customized, distortion-aware Gabor filters (PanoGabor filters).
• Design of a channel-wise and spatial-wise unidirectional fusion module (CS-UFM) to integrate PanoGabor filters and unify representations into ERP format.
• Implementation of a spherical gradient constraint to stabilize the orientation sensitivity of Gabor transforms.

Main Results:

• The proposed PGFuse framework demonstrates superior performance compared to state-of-the-art methods.
• Experimental results on three indoor 360 benchmarks validate the effectiveness of the approach.
• The method successfully generates effective and distortion-aware features for depth estimation.

Conclusions:

• PGFuse offers a robust solution for monocular 360 depth estimation by effectively managing image distortions.
• The framework enhances depth perception in 3D environments captured by 360 cameras.
• The developed techniques provide a significant advancement in the field of 360 image processing.