Related Experiment Video
Updated: Sep 18, 2025
07:13
Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
Published on: October 27, 2023
1.4K
SC-CoSF: Self-Correcting Collaborative and Co-Training for Image Fusion and Semantic Segmentation
Dongrui Yang1, Lihong Qiao1,2, Yucheng Shu1,2
1Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
Sensors (Basel, Switzerland)
|June 27, 2025
Summary
We introduce SC-CoSF, a novel framework for multimodal image fusion and semantic segmentation in autonomous systems. This approach enhances performance by jointly optimizing tasks, improving both fusion quality and segmentation accuracy.
Area of Science:
- Computer Vision
- Artificial Intelligence
- Robotics
Background:
- Multimodal image fusion and semantic segmentation are crucial for autonomous systems.
- Their interdependence is underexplored, leading to performance bottlenecks.
Purpose of the Study:
- To propose SC-CoSF, a novel coupled framework for joint optimization of image fusion and semantic segmentation.
- To address the underexplored interdependence and overcome performance limitations.
Main Methods:
- Utilizes a weight-sharing CNN encoder for implicit multimodal feature alignment and reduced parameters.
- Introduces a Self-correction and Collaboration Fusion Module (Sc-CFM) with Self-correction Long-Range Relationship Branch (Sc-LRB) and Self-correction Fine-Grained Branch (Sc-FGB).
- Employs Dual-branch Collaborative Recalibration (DCR), Interactive Context Recovery Mamba Decoder (ICRM), and Region Adaptive Weighted Reconstruction Decoder (ReAW).
Main Results:
- Demonstrates significant improvements in fusion quality compared to independently optimized baselines.
- Achieves superior segmentation accuracy through synergistic learning and cross-task feature refinement.
- Preserves critical edge textures and color contrasts while reducing feature redundancy.
Conclusions:
- SC-CoSF effectively leverages inter-task consistency for enhanced performance in multimodal image fusion and semantic segmentation.
- Joint optimization via end-to-end training enables gradient propagation for superior results.
- The proposed framework offers a promising solution for autonomous driving and robotic systems.
