Lightweight attention-aware fusion network based on state-space model for V-D-T salient object detection

With the popularization of various sensors, multimodal Salient Object Detection (SOD) methods including Visual-Depth-Thermal Salient Object Detection (V-D-T SOD) have made remarkable developments. However, most existing V-D-T SOD methods usually achieve accurate detection with expensive computational cost, which limits their development at the edge application. To address this issue, we develop a Lightweight Attention-Aware Fusion Network (LAANet) based on the State Space Model (SSM). Specifically, inspired by the cross-attention mechanism, the Cross Mamba Fusion Module (CMFM) is proposed to realize attention perception among shallow high-resolution multimodal features by exploiting SSM with linear complexity. For low-resolution deep features, Attention Perception Fusion Module (APFM) is proposed based on the self-attention mechanism to mine semantic cues and complement the potential information lost in SSM amnestic memory. In addition, to response the problem of excessive decoding parameters and efficiently achieve feature reconstruction,we design a lightweight decoder (LDB). It is consist of simple dilated convolutions and linear operations with only 0.37M parameter. Extensive experiments on the VDT2048 dataset show that our method achieves performance close to that of SOTA, while having smaller number of parameters (6.78M), lower model complexity (5.23G), and faster inference speed (23.7FPS when the input size is 384*384). The code is available at https://github.com/GZNU-WJT/LAANet.