CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing bi-modal salient object detection (SOD) methods, primarily using convolution, face performance ceilings due to local connectivity.
• Complex fusion structures in current methods limit effective cross-modal information integration for RGB-D and RGB-T data.

Purpose of the Study:

• To propose a novel transformer-based approach for salient object detection that overcomes the limitations of convolution operations.
• To introduce a new framework for global information alignment and transformation in multi-modal feature integration.

Main Methods:

• Developed the Cross-modal View-mixed Transformer (CAVER), a two-stream encoder-decoder framework utilizing a novel view-mixed attention mechanism.
• Implemented a parameter-free, patch-wise token re-embedding strategy to manage computational complexity associated with transformer tokens.
• Cascaded cross-modal integration units to create a top-down, transformer-based information propagation path for sequence-to-sequence context updates.

Main Results:

• CAVER demonstrated superior performance on standard RGB-D and RGB-T salient object detection datasets.
• The proposed transformer-based framework surpassed recent state-of-the-art methods in salient object detection tasks.
• Experimental results validated the effectiveness of the view-mixed attention and token re-embedding strategies.

Conclusions:

• The proposed CAVER framework offers a more effective approach to salient object detection by leveraging global information alignment through transformers.
• Transformer-based methods, when designed with efficient attention and tokenization strategies, can outperform traditional convolution-based approaches in multi-modal SOD.
• The study highlights the potential of global context modeling for enhancing cross-modal feature fusion in computer vision tasks.