Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360° Omnidirectional Image

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Salient Object Detection (SOD) identifies visually prominent objects in images.
• 360° omnidirectional images, common in virtual reality (VR), present unique challenges for SOD due to distortions and complex scenes.
• Existing SOD methods are not well-suited for the complexities of 360° imagery.

Purpose of the Study:

• To develop an effective method for salient object detection (SOD) in 360° omnidirectional images.
• To address the limitations of existing SOD techniques when applied to distorted and complex 360° scenes.
• To introduce a novel network architecture capable of handling the unique characteristics of omnidirectional imagery.

Main Methods:

• Proposes the Multi-Projection Fusion and Refinement Network (MPFR-Net).
• Simultaneously processes equirectangular projection (EP) images and four cube-unfolding (CU) images.
• Introduces a dynamic weighting fusion (DWF) module for adaptive feature integration and a filtration and refinement (FR) module for feature interaction optimization.

Main Results:

• The MPFR-Net demonstrates superior performance on two omnidirectional datasets compared to state-of-the-art methods.
• Qualitative and quantitative experimental results confirm the effectiveness of the proposed approach.
• The integration of EP and CU images, along with DWF and FR modules, significantly enhances SOD accuracy.

Conclusions:

• The MPFR-Net effectively tackles the challenges of salient object detection in 360° omnidirectional images.
• The proposed fusion and refinement strategy offers a significant advancement in processing complex, distorted imagery.
• This work provides a robust solution for SOD in VR and other applications utilizing 360° content.