CNN-Based Multimodal Human Recognition in Surveillance Environments.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biometrics

Background:

• Current human recognition research predominantly focuses on outdoor re-identification.
• Indoor human recognition research is limited, with existing methods often relying on facial recognition, which is unreliable due to camera angles.
• Indoor surveillance systems frequently capture partial body images, further complicating recognition.

Purpose of the Study:

• To propose a multimodal human recognition method combining face and body data for improved indoor accuracy.
• To address challenges of non-frontal face images and partial body captures in indoor environments.
• To enhance human recognition performance using deep convolutional neural networks (CNNs).

Main Methods:

• A multimodal approach utilizing both face and body recognition via deep convolutional neural networks (CNNs).
• Separate CNNs (VGG Face-16 and ResNet-50) are employed for face and body recognition.
• Score-level fusion using the Weighted Sum rule combines the results from separate CNNs to improve overall recognition performance.

Main Results:

• The proposed multimodal method significantly outperforms single-modality (face or body only) recognition.
• High recognition accuracy was achieved, with equal error rates of 1.52% on the DFB-DB1 database and 0.58% on the ChokePoint database.
• The method demonstrates superior performance compared to existing approaches for indoor human recognition.

Conclusions:

• The multimodal deep learning approach effectively overcomes limitations of single-modality recognition in indoor environments.
• Combining face and body recognition through score-level fusion enhances robustness and accuracy.
• This method offers a promising solution for reliable indoor human recognition in surveillance applications.