Related Experiment Video
Updated: Jul 8, 2025
Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
A visual questioning answering approach to enhance robot localization in indoor environments
Juan Diego Peña-Narvaez1, Francisco Martín2, José Miguel Guerrero2
1Intelligent Robotics Lab, Signal Theory, Communications, Telematics Systems, and Computation Department, International Doctoral School, Rey Juan Carlos University, Fuenlabrada, Spain.
This study introduces a new robot localization method using Visual Question Answering (VQA) for enhanced accuracy in complex environments. The approach improves robot navigation by integrating semantic understanding with mapping, leading to more reliable positioning.
Area of Science:
- Robotics
- Artificial Intelligence
- Computer Vision
Background:
- Robot localization in dynamic and complex environments like homes and offices presents significant challenges.
- Traditional mapping methods often struggle with accuracy and reliability in intricate spaces.
Purpose of the Study:
- To enhance robot localization precision in dynamic and complex environments.
- To integrate semantic insights into traditional mapping methods using Visual Question Answering (VQA).
- To develop a novel position hypothesis generation mechanism for improved localization.
Main Methods:
- Combines probabilistic approaches with Monte Carlo Localization (MCL) and Visual Language Models (VLMs).
- Integrates semantic mapping derived from VQA into the localization process.
- Formulates a novel hypothesis generation mechanism to assist localization.
Main Results:
- Achieved more robust robot localization compared to existing approaches.
- Outperformed state-of-the-art multi-hypothesis algorithms in position estimation and particle quality.
- Demonstrated high recovery rates from deliberate position alterations, showcasing robustness.
Conclusions:
- The proposed approach significantly enhances robot localization accuracy and reliability.
- Merging visual sensing, semantic mapping, and advanced localization techniques opens new possibilities for robot navigation.
- This work bridges visual perception, semantic understanding, and traditional mapping for more intelligent robot interaction with environments.
