MSAFusion: A Lightweight Multispectral Pedestrian Detection Network with Multi-Scale and Adaptive Feature Fusion

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Fusion

Background:

• Pedestrian detection is difficult in low light, high thermal contrast, and cluttered scenes.
• Existing lightweight fusion methods struggle with scale variations and modality bias, impacting accuracy.

Purpose of the Study:

• To develop an efficient and accurate lightweight RGB-thermal fusion pipeline for robust pedestrian detection.
• To improve feature representation and fusion control in multispectral pedestrian detection.

Main Methods:

• A stage-wise fusion pipeline incorporating Multi-scale Feature Refinement (MSFR) for pre-fusion enhancement.
• Utilized Cross-Modality Fusion Transformer (CFT) for improved semantic correspondence between RGB and thermal data.
• Implemented Adaptive Feature Recalibration (AFR) for post-fusion suppression of biased responses.

Main Results:

• The proposed method demonstrates a superior accuracy-efficiency trade-off compared to lightweight baselines.
• Consistent improvements were observed across multiple public RGB-thermal pedestrian detection benchmarks.
• The pipeline maintains a compact architecture suitable for real-time inference.

Conclusions:

• The developed lightweight fusion pipeline effectively addresses challenges in multispectral pedestrian detection.
• The integration of MSFR, CFT, and AFR modules enhances robustness and accuracy.
• The method offers a practical solution for real-time pedestrian detection in complex environments.