Convolutional neural net face recognition works in non-human-like ways.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biometrics

Background:

• Convolutional Neural Networks (CNNs) achieve state-of-the-art performance in pattern recognition.
• CNNs can be vulnerable to adversarial attacks, such as noise patterns.
• Existing research highlights CNNs' capabilities in face recognition tasks.

Purpose of the Study:

• To investigate the error patterns of commercial CNN-based face recognition systems.
• To compare the performance of CNNs with human participants in face matching.
• To understand how CNNs handle face variations related to sex and race.

Main Methods:

• Tested six commercial face recognition CNNs on standard face-matching tasks.
• Compared CNN performance against human participants.
• Analyzed discrepancies in matching decisions, particularly for faces with altered sex or race appearance.

Main Results:

• CNNs outperformed human participants on standard face-matching tasks.
• CNNs declared more matches for faces transformed to appear a different sex or race compared to human judgments.
• The best-performing CNNs showed near-perfect accuracy on human face-matching tasks while also exhibiting the most cross-sex/race misidentifications.
• Both humans and CNNs identified similar pairs of images as difficult, suggesting overlapping similarity spaces.

Conclusions:

• CNN face recognition systems exhibit distinct error patterns compared to humans, particularly concerning perceived sex and race.
• Despite differences in salience, CNNs and humans show some agreement in identifying difficult face-matching pairs.
• The findings suggest that while CNNs are highly accurate, their decision-making processes may not fully align with human perception of facial similarity across demographic variations.