Instant Panoramic Texture Mapping with Semantic Object Matching for Large-Scale Urban Scene Reproduction

Area of Science:

• Computer Graphics
• Virtual Reality
• Urban Modeling

Background:

• Image-based rendering (IBR) methods struggle with large-scale urban scenes, requiring extensive data or detailed geometry.
• Existing interactive IBR techniques for urban environments often lack high-quality street-level rendering capabilities.

Purpose of the Study:

• To develop a novel rendering system for real-time, photorealistic reproduction of large-scale urban scenes at street level.
• To enable free walk-through experiences in global urban streets using sparsely sampled data.

Main Methods:

• Utilizes panoramic texture mapping and simplified scene models from open databases.
• Extracts semantic information from street-view images to enhance rendering accuracy and performance.
• Incorporates real-time semantic 3D inpainting for occluded and untextured areas.

Main Results:

• Achieves enhanced rendering accuracy and improved performance time compared to existing methods.
• Demonstrates effective coverage of large-scale scenes using sparse panoramic images.
• Successfully handles dynamic viewpoint changes and occlusions through semantic inpainting.

Conclusions:

• The proposed system offers an effective solution for high-quality, real-time street-level rendering of urban environments.
• Semantic information integration is key to improving IBR performance and accuracy for urban scenes.
• The system provides a foundation for immersive virtual exploration of cities.