IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Current video understanding models struggle to infer human intent, a key aspect of social intelligence.
• Existing benchmarks may overestimate model capabilities due to dataset biases.
• Bridging the gap between visual observation and intent reasoning is crucial for advanced AI.

Purpose of the Study:

• Introduce a novel task, IntentQA, and a large-scale dataset for video intent reasoning.
• Develop a robust evaluation methodology beyond simple accuracy, addressing dataset biases.
• Propose an explainable framework for context-aware video intent reasoning.

Main Methods:

• Created the IntentQA dataset and five contrast sets using Large Language Models (LLMs).
• Developed the X-CaVIR (eXplainable Context-aware Video Intent Reasoning) framework.
• Integrated Situational, Contrastive, and Commonsense Contexts using modules like Video Query Language (VQL) and Contrastive Learning.
• Employed a transparent pipeline synergizing video captions with VQA model outputs for LLM integration.

Main Results:

• The X-CaVIR framework demonstrated superior performance against state-of-the-art baselines.
• The proposed contrast sets and metric effectively evaluated model robustness.
• The transparent LLM integration enhanced performance and provided explicit interpretability.
• Experiments confirmed the effectiveness of individual components and overall framework stability.

Conclusions:

• The IntentQA dataset and X-CaVIR framework advance video understanding by focusing on human intent reasoning.
• Robust evaluation methods are essential to mitigate dataset biases and accurately assess AI capabilities.
• Explainable AI approaches, like X-CaVIR, are vital for transparent and reliable video analysis.