Guiding visual attention in deep convolutional neural networks based on human eye movements

Area of Science:

• Computational Neuroscience
• Computer Vision
• Artificial Intelligence

Background:

• Deep Convolutional Neural Networks (DCNNs) show parallels with the human ventral visual pathway for object recognition.
• Recent deep learning advances may reduce this biological similarity, posing challenges for computational neuroscience.
• Previous work explored biologically inspired architectures; this study uses a data-driven approach.

Purpose of the Study:

• To investigate a purely data-driven method for guiding DCNN visual attention using human eye-tracking data.
• To assess if manipulating training data to mimic or contrast human visual focus impacts DCNN object recognition.
• To evaluate the effects on human-likeness and implications for face detection theories.

Main Methods:

• Human eye-tracking data was used to modify training images, guiding DCNNs' visual attention.
• Manipulation types included standard, human-like, and non-human-like attention focus.
• GradCAM saliency maps were compared against human eye-tracking data for validation.

Main Results:

• Guided focus manipulation effectively directed DCNN attention, particularly in the negative (non-human-like) direction.
• Non-human-like attention models focused on significantly different image regions compared to humans.
• Effects were category-specific, influenced by animacy and face presence, and emerged post-feedforward processing, notably impacting face detection.

Conclusions:

• While the data-driven attention guidance influenced DCNNs and face detection, it did not significantly increase human-likeness.
• The findings suggest potential applications for overt visual attention in DCNNs and offer insights into face detection mechanisms.