Context-Aware Graph Inference With Knowledge Distillation for Visual Dialog.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Visual dialog requires understanding complex semantic dependencies between visual and textual contexts.
• Modeling context-aware relational inference in graphical models with sparse nodes and unknown structures is challenging.

Purpose of the Study:

• To propose a novel context-aware graph (CAG) neural network for visual dialog.
• To enhance fine-grained relational reasoning using object-level dialog-historical co-reference nodes.
• To introduce CAG-Distill, a visual-aware knowledge distillation mechanism to mitigate linguistic bias.

Main Methods:

• Developed a context-aware graph (CAG) neural network with adaptive top-K message passing for iterative graph structure updates.
• Implemented dynamic relations for each node, selecting the most relevant neighbors for inference.
• Proposed CAG-Distill for knowledge distillation using image-only visual cues to regularize dialog-historical contextual awareness.

Main Results:

• CAG and CAG-Distill significantly outperformed comparative methods on VisDial v0.9 and v1.0 datasets.
• The proposed methods demonstrated improved contextual awareness and relational inference.
• Visualization results confirmed the interpretability of the graph inference approach.

Conclusions:

• The novel CAG neural network effectively models context-aware relational inference in visual dialog.
• CAG-Distill successfully mitigates linguistic bias by leveraging visual information.
• The proposed approach offers a promising direction for advancing visual dialog systems.