TCCDNet: A Multimodal Pedestrian Detection Network Integrating Cross-Modal Complementarity with Deep Feature Fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multimodal pedestrian detection is crucial for complex scenarios.
• Leveraging infrared and visible light complementarity enhances performance.
• Challenges remain in cross-modal fusion and inter-modal complementarity exploration.

Purpose of the Study:

• To propose TCCDNet, a novel network for multimodal pedestrian detection.
• To enhance feature extraction and fusion using cross-modal complementarity.
• To improve object localization accuracy and feature discriminability.

Main Methods:

• Developed the efficient multi-scale attention C2f (EMAC) backbone for feature weighting and fusion.
• Introduced the cross-modal complementarity (CMC) module with channel and spatial attention.
• Incorporated a deep semantic fusion module (DSFM) using cross-attention for deep semantic fusion.

Main Results:

• TCCDNet achieved 7.87% MR^2 on the KAIST dataset, outperforming YOLOv8 by 3.83%.
• Attained 83.8% mAP50 on FLIR ADAS and 97.3% mAP50 on LLVIP, surpassing baselines by 3.6% and 1.9% respectively.

Conclusions:

• TCCDNet demonstrates significant advancements in multimodal pedestrian detection.
• The proposed fusion mechanisms effectively utilize inter-modal complementarity.
• The network shows superior performance across multiple benchmark datasets.