A Large-Scale Virtual Dataset and Egocentric Localization for Disaster Responses.

Area of Science:

• Computer Vision
• Robotics
• Disaster Response

Background:

• Increasing demand for visual observation in disaster response.
• Shortage of datasets for disaster scenarios hindering progress in computer vision and robotics.
• Need for realistic data to train and evaluate AI models for rescue and safety.

Purpose of the Study:

• To present the first large-scale synthetic dataset of egocentric viewpoints for disaster scenarios.
• To enable advancements in computer vision and robotics for disaster response.
• To develop and evaluate robust egocentric localization methods for disaster environments.

Main Methods:

• Simulation of pre- and post-disaster scenarios with drastic appearance changes (fire, earthquakes).
• Creation of over 300K high-resolution stereo image pairs with comprehensive annotations (semantic label, depth, optical flow, surface normal, camera poses).
• Augmentation of realistic disaster scenes using 3D models and physically-based graphics.
• Training and evaluation of state-of-the-art computer vision methods on the dataset.
• Proposal of a novel convolutional neural network-based egocentric localization method robust to appearance and layout changes.

Main Results:

• The dataset facilitates training and evaluation of computer vision tasks in disaster scenarios.
• State-of-the-art methods show improved recognition of disaster situations and reliable results.
• The proposed egocentric localization method demonstrates robustness to drastic appearance and layout changes.
• Experimental results confirm reliable camera pose predictions even in significantly altered conditions.

Conclusions:

• The developed synthetic dataset is crucial for advancing computer vision and robotics in disaster response.
• The proposed egocentric localization method offers reliable performance in challenging, dynamic disaster environments.
• This work provides a valuable resource and a robust solution for improving safety and rescue operations through AI.