Measuring Human Perception to Improve Open Set Recognition.

Area of Science:

• Computer Vision
• Cognitive Psychology
• Machine Learning

Background:

• Human object recognition capabilities significantly exceed current open set recognition algorithms.
• Visual psychophysics provides a method to measure human perception and reaction times, offering insights into object classification.
• Managing novelty and unknown data is a critical challenge for existing recognition systems.

Purpose of the Study:

• To leverage human reaction time data to improve the performance of artificial intelligence algorithms in open set recognition tasks.
• To design and validate a novel psychophysical loss function that aligns deep network behavior with human perception.
• To enhance the ability of algorithms to distinguish between known and unknown object classes, particularly in data-scarce scenarios.

Main Methods:

• Conducted a large-scale behavioral experiment collecting over 200,000 human reaction time measurements for object recognition.
• Developed a new psychophysical loss function that integrates human behavioral data (reaction times) into deep neural network training.
• Trained Multi-Scale DenseNets using the proposed loss function on ImageNet data for open set recognition evaluation.

Main Results:

• Human reaction times vary significantly at the sample level, indicating nuanced object recognition behavior.
• The novel psychophysical loss function led to substantial improvements: +6.02% top-1 validation accuracy, +9.81% top-1 test accuracy on known samples, and +33.18% top-1 test accuracy on unknown samples.
• The proposed method outperformed 10 existing open set recognition algorithms across multiple performance metrics.

Conclusions:

• Incorporating human psychophysical data, specifically reaction times, is an effective strategy for enhancing open set recognition in deep learning models.
• The developed psychophysical loss function enables models to better manage novelty and improve accuracy, even with limited labeled data.
• This approach represents a significant advancement in bridging the gap between human and machine visual recognition capabilities.