Int3DNet: Scene-Motion Cross Attention Network for 3D Intention Prediction in Mixed Reality.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Robotics

Background:

• Intention prediction is crucial for seamless Mixed Reality (MR) experiences.
• Current methods often rely on explicit object recognition, limiting robustness.
• Anticipating user actions proactively enhances user experience and reduces interaction delays.

Purpose of the Study:

• To develop a scene-aware network (Int3DNet) for direct 3D intention area prediction.
• To enable robust human intention prediction without explicit object-level perception.
• To improve human-MR system interaction through proactive intention processing.

Main Methods:

• Int3DNet utilizes scene geometry and head-hand motion cues.
• A cross-attention fusion mechanism combines sparse motion data with scene point clouds.
• The network directly interprets spatial intention within the scene context.

Main Results:

• Int3DNet demonstrated consistent performance across various time horizons (up to 1500 ms).
• The method outperformed baseline approaches on MoGaze and CIRCLE datasets.
• Effective prediction was achieved even in diverse and previously unseen scenes.

Conclusions:

• Int3DNet provides reliable 3D intention areas using head-hand motion and scene geometry.
• The approach enables seamless human-MR interaction via proactive intention area processing.
• The method's usability was confirmed through a visual question answering (VQA) demonstration.