Salient Object Detection From Arbitrary Modalities.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Existing salient object detection (SOD) algorithms struggle with dynamic input changes.
• Current SOD methods require specific training for each input modality, increasing costs.
• Lack of generalized SOD solutions limits real-world applicability.

Purpose of the Study:

• Introduce a novel SOD task: Arbitrary Modality SOD (AM SOD).
• Address the challenge of dynamically changing input modality types and numbers in SOD.
• Develop a generalized SOD algorithm adaptable to various input configurations.

Main Methods:

• Propose a Modality Switch Network (MSN) for AM SOD.
• Utilize a Modality Switch Feature Extractor (MSFE) with modality indicators for feature extraction.
• Employ a Dynamic Fusion Module (DFM) with a Transformer structure for adaptive feature fusion.
• Construct the AM-XD dataset to support AM SOD research.

Main Results:

• The proposed AM SOD method effectively handles changes in input modality type and number.
• Experiments demonstrate robust salient object detection performance across varied inputs.
• The MSN approach shows significant improvements in adaptability and generalization.

Conclusions:

• AM SOD offers a flexible and cost-effective solution for salient object detection.
• The developed MSN framework provides a robust approach to multimodal SOD.
• The AM-XD dataset facilitates future research in dynamic and multimodal SOD.