Anticipating Where People will Look Using Adversarial Networks.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Gaze prediction typically focuses on current video frames.
• Anticipating future gaze is crucial for advanced human-computer interaction and robotics.
• Existing models struggle to predict gaze beyond the immediate frame.

Purpose of the Study:

• To introduce and address the novel problem of gaze anticipation on future video frames.
• To propose a new generative adversarial network-based model, Deep Future Gaze (DFG), for this task.
• To evaluate DFG's performance against state-of-the-art methods on diverse video datasets.

Main Methods:

• Developed Deep Future Gaze (DFG), a generative adversarial network with two pathways: DFG-P for prior maps and DFG-G for future frame generation.
• DFG-G utilizes a two-stream spatial-temporal convolution (3D-CNN) generator to create synthetic future frames, incorporating semantic and motion information.
• A discriminator network distinguishes real from synthetic frames, enhancing the generator's capabilities. Gaze prediction is performed on generated frames.

Main Results:

• DFG significantly outperformed all competitive baselines on publicly available egocentric and third-person video datasets.
• The model demonstrated superior performance in gaze prediction on current frames compared to state-of-the-art methods.
• Ablation studies confirmed the effectiveness of DFG's dual-pathway architecture.

Conclusions:

• Deep Future Gaze (DFG) effectively solves the problem of gaze anticipation on future frames.
• The proposed method advances the state-of-the-art in both future and current gaze prediction.
• DFG offers promising applications in areas requiring proactive understanding of human attention.