Attention Fusion for One-Stage Multispectral Pedestrian Detection

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Multispectral pedestrian detection combines color and thermal data for improved performance in low illumination.
• Deep Convolutional Neural Networks (CNNs) are crucial for leveraging complementary information from multispectral streams.
• Existing methods often use complex two-stage detectors, necessitating simpler, efficient alternatives.

Purpose of the Study:

• To adapt and improve a one-stage YOLOv4 detector for efficient multispectral pedestrian detection.
• To propose a novel Multispectral Channel Feature Fusion (MCFF) module for adaptive feature integration.
• To design and evaluate various fusion architectures (Early, Halfway, Late, Direct) for optimal performance.

Main Methods:

• Replaced a two-stage fast-RCNN with a one-stage YOLOv4 for direct bounding box prediction.
• Developed the MCFF module to integrate color and thermal stream features based on illumination.
• Implemented and compared four fusion architectures (Early, Halfway, Late, Direct) using the MCFF module.

Main Results:

• The Halfway Fusion architecture demonstrated superior performance among all tested fusion strategies.
• The MCFF module effectively adapted fused features from both color and thermal modalities.
• Achieved log-average miss rates of 4.91% (color) and 23.14% (thermal) on benchmark datasets.

Conclusions:

• The proposed MCFF module and Halfway Fusion architecture offer an efficient and effective solution for multispectral pedestrian detection.
• This approach significantly enhances pedestrian detection capabilities, particularly in challenging low-illumination environments.
• The study provides a valuable contribution to the field of computer vision for autonomous systems.