Real-Time View Planning for Unstructured Lumigraph Modeling.

Area of Science:

• Computer Vision
• Computer Graphics
• Mixed Reality

Background:

• Real-time generation of 3D scene representations from image streams is crucial for mixed reality applications like telepresence and as-built documentation.
• Conventional methods often rely on extensive texture optimization in structure-from-motion pipelines, which are not suitable for incremental view selection.

Purpose of the Study:

• To propose an algorithm for generating unstructured lumigraphs in real-time from a continuous image stream.
• To address the challenge of incremental view selection under strict storage and transmission constraints.

Main Methods:

• Formulated an online variant of the view-planning problem, considering previously captured scene parts and future lumigraph sample distribution.
• Regularized scene structure using a grid, defining a coverage metric based on spatial and angular resolution.
• Employed a greedy approach to select incoming views that enhance coverage.

Main Results:

• Achieved visually appealing results on synthetic and real scenes.
• Demonstrated real-time performance, with frame rates ranging from 3Hz to 100Hz depending on configuration.
• Successfully generated unstructured lumigraphs incrementally from image streams.

Conclusions:

• The proposed algorithm effectively generates real-time unstructured lumigraphs by optimizing view selection incrementally.
• The method provides a practical solution for mixed reality and related applications with limited data handling capabilities.