A Salient Object Detection Network Enhanced by Nonlinear Spiking Neural Systems and Transformer

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Existing deep learning methods for RGB-D Salient Object Detection (SOD) struggle with cross-modal feature fusion, depth noise sensitivity, and limited generalization.
• These challenges hinder accurate saliency estimation in complex visual data.

Purpose of the Study:

• To introduce TranSNP-Net, an innovative deep learning model for RGB-D SOD.
• To address limitations in feature fusion, depth noise handling, and model generalization.

Main Methods:

• Integration of Nonlinear Spiking Neural P (NSNP) systems with Transformer networks.
• Utilizing an enhanced feature fusion module (SNPFusion) and attention mechanism for cross-modal fusion.
• Employing a fine-tuned Swin Transformer backbone for improved generalization.
• Implementing a hierarchical feature decoder (SNP-D) for enhanced accuracy in noisy depth scenes.

Main Results:

• TranSNP-Net achieved superior performance across six RGB-D benchmark datasets.
• Mean scores for S-measure, F-measure, E-measure, and MEA were 0.9328, 0.9356, 0.9558, and 0.0288, respectively.
• Outperformed 14 leading SOD methods.

Conclusions:

• TranSNP-Net effectively fuses RGB and depth information, demonstrating robust performance.
• The model shows significant improvements in generalization and accuracy, particularly in challenging conditions with depth noise.
• TranSNP-Net represents a substantial advancement in RGB-D salient object detection.