Improved likelihood ratios for face recognition in surveillance video by multimodal feature pairing

Area of Science:

• Forensic Science
• Computer Vision
• Biometrics

Background:

• Facial recognition in surveillance is crucial for forensic and security applications.
• Variations in pose, illumination, and expression challenge accuracy.
• Manual comparison methods lack standardization, impacting evidence reliability.

Purpose of the Study:

• To develop an enhanced images-to-video facial recognition approach for forensic scenarios.
• To assess the reliability and accuracy of different facial recognition models and preprocessing techniques.

Main Methods:

• Proposed an enhanced images-to-video recognition approach using facial images with pose and quality attributes.
• Utilized diverse datasets (ENFSI 2015, SCFace, XQLFW, ChokePoint, ForenFace) for validation.
• Evaluated three models (ArcFace, FaceNet, QMagFace) using log-likelihood ratio cost (C) and calibration methods.

Main Results:

• Prioritizing high-quality frames and aligning attributes with reference images optimized recognition performance.
• The best approach yielded C values comparable to using the top 25% of frames.
• A combined embedding weighted by frame quality was the second-best method.
• Super-resolution preprocessing with CodeFormer unexpectedly increased C, reducing evidence reliability.

Conclusions:

• Enhanced facial recognition by incorporating image attributes and quality significantly improves forensic video analysis.
• Careful selection of high-quality frames and attribute alignment are key for reliable facial recognition.
• Super-resolution techniques like CodeFormer should be used cautiously in forensic applications due to potential negative impacts on accuracy.