A study on infrared-visible fusion multimodal object detection algorithm based on cross-modal information bottleneck...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Sensor Fusion

Background:

• Multimodal object detection is crucial for navigating complex environments.
• Existing methods struggle with modality redundancy and feature misalignment.

Purpose of the Study:

• To develop a novel multimodal fusion detection framework.
• To address limitations in modality redundancy suppression and feature alignment.

Main Methods:

• Proposed a framework integrating Cross-modal Information Bottleneck (CIB) and Minimum Redundancy Transformation (MRT).
• CIB uses a compress-decompose-reconstruct pathway for shared semantics.
• MRT applies sparse transformations to reduce redundancy and enhance boundary awareness.
• Implemented a dual-phase training strategy (modality isolation and fusion).

Main Results:

• Improved mAP on KAIST nighttime scenario from 42.8% to 44.1%.
• Achieved 80.0% AP@75 in LLVIP low-light conditions, surpassing state-of-the-art by 2.4%.
• Demonstrated consistent robustness against occlusion and illumination variations.

Conclusions:

• The proposed framework effectively suppresses modality redundancy and aligns features.
• The approach shows significant potential for reliable multimodal perception in adverse conditions.