SHC: Deeply Activating Human-Like Cognitive Ability for Visual Question Answering.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Computer Vision

Background:

• Human cognition relies on complex information processing, but current AI models often study processes in isolation.
• Large language models (LLMs) have advanced language understanding, yet visual information processing remains a challenge.

Purpose of the Study:

• To propose a Systematic Human-like Cognitive (SHC) method for visual question answering (VQA).
• To enhance multi-modal understanding by systematically modeling human cognitive processes.

Main Methods:

• SHC models human cognition through three modules: Hierarchical Perception (HPM), Semantic Refinement (SRM), and Dynamic Reasoning (DRM).
• HPM extracts hierarchical features, SRM refines semantics using cascaded attention, and DRM fuses features for unified representation.
• The fused representation is used with text input in an LLM for reasoning.

Main Results:

• SHC achieves competitive performance on VQA benchmarks (VQA-v2, Text-VQA, GQA, ScienceQA).
• It shows strong results on multimodal benchmarks (POPE, MMB, MME, MM-Vet).
• SHC demonstrates superior fine-grained visual perception and outperforms larger models on some tasks.

Conclusions:

• The SHC method effectively simulates human cognitive mechanisms for improved VQA.
• SHC offers a promising approach for advancing multi-modal understanding and visual perception in AI.