Out-of-Sight Embodied Agents: Multimodal Tracking, Sensor Fusion, and Trajectory Forecasting.

Area of Science:

• Computer Vision
• Robotics
• Autonomous Systems

Background:

• Existing trajectory prediction models struggle with noisy sensor data and out-of-sight agents.
• Sensor noise and occlusions in real-world applications like autonomous driving pose safety risks.
• Accurate trajectory prediction is crucial for robust performance in various autonomous systems.

Purpose of the Study:

• To improve trajectory prediction for out-of-sight agents using noisy sensor observations.
• To introduce a Vision-Positioning Denoising Module for unsupervised denoising of sensor signals.
• To expand the Out-of-Sight Trajectory Prediction task to include both pedestrians and vehicles.

Main Methods:

• Developed an improved Vision-Positioning Denoising Module utilizing camera calibration.
• Established vision-position correspondence to mitigate lack of direct visual cues.
• Expanded the Out-of-Sight Trajectory Prediction (OOSTraj) setting to include vehicles alongside pedestrians.

Main Results:

• Achieved state-of-the-art results in both trajectory denoising and prediction on Vi-Fi and JRDB datasets.
• Demonstrated clear performance gains over prior baselines and classical denoising techniques like Kalman filtering.
• Established a stronger benchmark by adapting recent trajectory prediction models to the OOSTraj setting.

Conclusions:

• The proposed method effectively denoises noisy sensor trajectories and predicts future paths of out-of-sight agents.
• Incorporating vision-positioning projection offers a novel approach for denoising sensor trajectories.
• This work opens new research avenues in trajectory prediction for autonomous systems facing real-world sensing challenges.