Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Multi-human detection and tracking in indoor surveillance presents significant challenges.
• Occlusions, varying illumination, and complex interactions hinder performance.
• Existing methods often struggle with these dynamic environmental factors.

Purpose of the Study:

• To investigate the efficacy of low-level sensor fusion for enhancing multi-human detection and tracking.
• To compare the performance of different input features and deep learning architectures.
• To determine optimal sensor fusion strategies for indoor surveillance applications.

Main Methods:

• A custom dataset was generated using a neuromorphic vision sensor (NVS) camera in an indoor setting.
• Experiments involved various image features and deep learning networks.
• A multi-input fusion strategy was employed to optimize for overfitting and analyze input feature importance.

Main Results:

• Significant differences were observed in the performance of input features across optimized deep learning backbones.
• Under low-data conditions, event-based frames from NVS were found to be superior.
• With higher data availability, the combination of grayscale and optical flow features yielded the best results.

Conclusions:

• Sensor fusion, particularly combining grayscale and NVS data, shows significant potential for improving multi-human tracking in indoor surveillance.
• The optimal feature selection is data-dependent, highlighting the need for adaptive strategies.
• Further research is recommended to validate and extend these findings.