Multispectral image fusion based pedestrian detection using a multilayer fused deconvolutional single-shot detector.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multispectral image fusion enhances pedestrian detection across varied lighting conditions.
• Existing methods struggle with small pedestrian detection and high computational costs.

Purpose of the Study:

• To propose a multilayer fused deconvolutional single-shot detector for improved multispectral pedestrian detection.
• To address limitations in small pedestrian detection and computational efficiency.

Main Methods:

• A two-stream convolutional module (TCM) extracts features from multispectral images.
• A multilayer fused deconvolutional module (MFDM) fuses features at multiple deconvolutional layers using fusion blocks.
• This approach combines high-level semantic and low-level detailed features for enhanced representational power.

Main Results:

• Achieved 81.82% average precision (AP) on a new small-sized multispectral pedestrian dataset.
• On the KAIST dataset, attained 97.36% AP and 20 fps, surpassing state-of-the-art by 6.82% AP and tripling detection speed.
• Demonstrated superior performance on two public multispectral pedestrian datasets.

Conclusions:

• The proposed multilayer fusion strategy effectively utilizes multispectral information.
• Fusion blocks reduce computational cost and redundant parameters, enhancing efficiency.
• The method offers a significant advancement in accurate and fast multispectral pedestrian detection, particularly for small targets.