Study on salient object segmentation based on depth information guidance and SAM low-rank adaptation fine-tuning

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Accurate salient object segmentation is vital for computer vision tasks like autonomous driving.
• RGB-D data improves saliency detection but faces challenges with sensor dependency and data fusion.
• Existing methods struggle with complex scenes and integrating multi-modal information effectively.

Purpose of the Study:

• To develop an innovative salient object segmentation method integrating the Segment Anything Model (SAM), depth information, and cross-modal attention.
• To enhance segmentation accuracy and robustness in diverse and challenging visual scenes.
• To reduce dependency on depth sensors and improve the fusion of RGB and depth data.

Main Methods:

• Leveraging the Segment Anything Model (SAM) for robust feature extraction.
• Integrating a pre-trained depth estimation network for geometric information capture.
• Employing cross-modal attention mechanisms for dynamic RGB and depth feature fusion.
• Utilizing lightweight LoRA fine-tuning and a UNet decoder for computational efficiency and precise boundary detail preservation.

Main Results:

• The proposed method demonstrates significant improvements over existing approaches on five benchmark datasets.
• Achieved superior performance in MaxF, MAE, and S-measure metrics, especially in complex scenarios.
• Showcased enhanced robustness for segmenting small targets, multiple objects, and scenes with intricate backgrounds.

Conclusions:

• The developed method effectively enhances depth-guided RGB salient object segmentation, overcoming depth sensor limitations.
• The approach offers novel insights into cross-modal information fusion for computer vision applications.
• This work contributes to advancing related technologies and their diversification through improved segmentation capabilities.