Many Ways to Think

Area of Science:

• Cognitive Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Thinking, collaboration, and communication integrate body, mind, language, and environmental interactions.
• Embodied actions like gestures and graphics serve as direct representations of thought, surpassing verbal communication.
• These interactions are often internal, unique, and difficult to decompose, influencing meaning in real-time.

Purpose of the Study:

• To explore the intricate relationship between embodied cognition, gestures, graphics, and human thought processes.
• To highlight the challenges these complex, real-time interactions present for artificial intelligence (AI).

Main Methods:

• Analysis of embodied actions (gestures) and visual representations (graphics) in expressing and shaping thought.
• Examination of the unique, internal, and dynamic nature of meaning-making through these modalities.

Main Results:

• Gestures and graphics can represent complex ideas, environments, and action sequences more directly than words.
• Engaging with gestures and graphics alters cognitive processes for both creators and viewers.
• Shared and individual meanings emerge dynamically from real-time interactions with these modalities.

Conclusions:

• The embodied and multimodal nature of human thought, involving gestures and graphics, is not easily captured by current AI.
• Future multi-modal AI must better account for the dynamic, internal, and context-dependent aspects of human cognition and communication.