Can Negation Be Depicted? Comparing Human and Machine Understanding of Visual Representations

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Computer Vision

Background:

• A common assumption is that visual representations lack negation.
• This study investigates if humans and AI can perceive negation in real-world images like photographs and manga.

Purpose of the Study:

• To empirically test the ability of humans and deep neural networks to recognize negation in visual representations.
• To explore the role of common sense knowledge in human visual negation understanding.
• To inform the development of AI with enhanced logical concept comprehension.

Main Methods:

• Collected human-generated captions for images to identify negation phrases.
• Assessed human and machine learning model performance in classifying images expressing negation.
• Analyzed differences in human and AI capabilities in visual negation detection.

Main Results:

• Human analysis of image captions provided evidence for recognizing visual negation.
• Humans demonstrated a moderate ability to classify novel images depicting negation.
• Machine learning models performed at chance level, significantly below human performance.

Conclusions:

• Human ability to perceive visual negation is linked to background common sense knowledge.
• Current AI models struggle with visual negation compared to humans.
• Findings offer insights into human cognition and guide AI development for logical concept understanding.