UTDNet: A unified triplet decoder network for multimodal salient object detection.

Area of Science:

• Computer Vision
• Image Processing

Background:

• Salient Object Detection (SOD) is crucial in computer vision.
• Multimodal data (RGB, Depth, Thermal) enhances SOD.
• Existing methods are limited to specific modalities (RGB-D or RGB-T) or require dataset-specific fine-tuning, hindering practical use.

Purpose of the Study:

• To propose an end-to-end Unified Triplet Decoder Network (UTDNet) for both RGB-T and RGB-D SOD tasks.
• To address challenges in unified multimodal SOD: accurate salient object detection and a single network for multiple modalities.
• To improve the practical deployment of SOD in real-world applications.

Main Methods:

• Developed a multi-scale feature extraction unit for enriched contextual information.
• Introduced an efficient fusion module for exploring cross-modality complementary information.
• Implemented a triplet decoder with hierarchical deep supervision loss for salient object detection.
• Utilized a continual learning method with Elastic Weight Consolidation (EWC) regularization to unify multimodal SOD tasks without additional parameters.

Main Results:

• The UTDNet effectively detects and segments salient objects across different modalities.
• The proposed continual learning approach successfully unifies RGB-T and RGB-D SOD tasks within a single network.
• The triplet decoder architecture facilitates adaptation to various multimodal SOD tasks by separating task-specific and task-invariant information.
• Extensive comparisons showed UTDNet outperforms 26 recent RGB-T and RGB-D SOD methods.

Conclusions:

• UTDNet offers a superior, unified solution for multimodal Salient Object Detection.
• The network's design enhances accuracy and adaptability for practical SOD applications.
• This work advances the field by enabling efficient and versatile multimodal SOD.