Shape Guides Visual Pretense.

Area of Science:

• Cognitive Psychology
• Computer Vision
• Human-Computer Interaction

Background:

• Pretense, where one object stands for another, is a fundamental aspect of human cognition, observed in play, education, and storytelling.
• The cognitive mechanisms underlying visual pretense, particularly the features humans prioritize when mapping one object onto another, remain incompletely understood.

Purpose of the Study:

• To investigate the specific features that guide human visual pretense.
• To compare human pretense preferences with the capabilities of modern multi-modal vision models.

Main Methods:

• A behavioral approach involving four studies with 716 participants to identify systematic preferences in visual pretense.
• A computational approach comparing human data with performant multi-modal vision models.

Main Results:

• Human visual pretense systematically favors spatial and physical alignment, particularly shape similarity, over surface features like color.
• People consistently align the subpart structures of real and pretend objects.
• Current multi-modal vision models do not adequately account for these human pretense preferences, likely due to their focus on surface features.

Conclusions:

• Human visual pretense relies heavily on geometric and structural properties, not superficial attributes.
• There is a significant divergence between human visual pretense strategies and the current computational approaches in multi-modal AI.
• Future AI development should incorporate spatial and structural reasoning to better model human-like object understanding and pretense.