An Evaluation of the Pedestrian Classification in a Multi-Domain Multi-Modality Setup.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Fusion

Background:

• Pedestrian classification is crucial for autonomous systems.
• Existing methods for visible and far-infrared (FIR) domains are difficult to compare due to dataset limitations.
• A unified dataset and comparative analysis are needed for robust pedestrian detection.

Purpose of the Study:

• To introduce a public dataset (RIFIR) for comparing pedestrian classification across visible and FIR domains.
• To evaluate state-of-the-art features (ISS, LBP, LGP, HOG) in a multi-modality setup (intensity, depth, motion).
• To compare performance across different light spectrum domains and modalities for pedestrian detection.

Main Methods:

• Collected a new public dataset (RIFIR) with synchronized visible and FIR images from a moving vehicle.
• Extracted and compared intensity self-similarity (ISS), local binary patterns (LBP), local gradient patterns (LGP), and histogram of oriented gradients (HOG) features.
• Evaluated features across visible and FIR domains, considering intensity, depth, and motion modalities.

Main Results:

• FIR domain generally outperformed the visible domain for pedestrian classification.
• Features from FIR and visible domains were found to be highly complementary.
• The best pedestrian detection performance was achieved through multi-domain, multi-modality, and multi-feature fusion.

Conclusions:

• The RIFIR dataset enables standardized comparison of pedestrian detection algorithms.
• Multi-modal and multi-domain fusion significantly enhances pedestrian classification accuracy.
• Far-infrared imaging offers advantages for pedestrian detection, especially when combined with visible light data.