Integrating Neural Radiance Fields End-to-End for Cognitive Visuomotor Navigation

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Existing navigation systems often rely on modular designs and explicit scene representations.
• Integrating implicit spatial representations like Neural Radiance Fields (NeRF) for embodied decision-making remains a challenge.

Purpose of the Study:

• To propose an end-to-end visuomotor navigation framework leveraging NeRF for spatial cognition.
• To introduce a novel mechanism for extracting decision-critical information from NeRF memory for embodied agents.
• To enable intelligent behavioral patterns and improve navigation performance without environmental priors.

Main Methods:

• Developed an end-to-end visuomotor navigation framework integrating NeRF for online spatial memory construction.
• Introduced Structural Radiance Attention (SRA) to extract historical insights from NeRF's implicit structure.
• Trained and tested the system on image-goal navigation tasks and deployed on a mobile robot.

Main Results:

• The NeRF-based framework significantly outperforms existing navigation models in image-goal tasks.
• Structural Radiance Attention (SRA) demonstrably improves scene and task understanding by leveraging NeRF memory.
• The system exhibits enhanced exploratory awareness and real-time performance in real-world robotic navigation.

Conclusions:

• The proposed end-to-end framework effectively integrates NeRF for cognitive navigation, eliminating the need for modular designs.
• SRA enhances agent's ability to grasp complex scenes and task objectives, leading to intelligent behaviors.
• The system demonstrates robust cognitive capabilities and real-time performance in real-world applications.