Cross-Modal Object Detection Based on Content-Guided Feature Fusion and Self-Calibration

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional transformers have limitations in local attention, hindering feature representation and accuracy in cross-modal object detection.
• Deep features can degrade through convolutional layers, leading to loss of critical object details.

Purpose of the Study:

• To develop an advanced dual-backbone cross-modal object detection model.
• To overcome limitations in local attention and feature degradation in existing models.

Main Methods:

• A parallel network was introduced in the backbone for simultaneous multi-modal processing.
• A content-guided fusion (CGF) module combined transformer and convolution for global and local feature extraction.
• An adaptive calibration fusion (ACF) module merged shallow and deep features to retain fine-grained details.

Main Results:

• The model achieved mAP50 of 96.4 and mAP95 of 63.8 on the LLVIP dataset.
• On the M3FD dataset, the model reached mAP50 of 83.7 and mAP95 of 56.6.
• Outperformed baseline and state-of-the-art methods in detection accuracy.

Conclusions:

• The proposed dual-backbone model effectively enhances cross-modal object detection.
• The novel fusion modules improve feature representation and detection accuracy in complex scenarios.
• Demonstrated robust performance across various environments for cross-modal object detection tasks.