Blind Augmentation: Calibration-Free Camera Distortion Model Estimation for Real-Time Mixed-Reality Consistency

Area of Science:

• Computer Vision
• Augmented Reality
• Image Processing

Background:

• Real camera footage contains noise, motion blur (MB), and depth of field (DoF) effects.
• Modeling these effects is crucial for visually integrating virtual content into live video feeds for augmented reality (AR).
• Existing methods often require complex camera calibration and slow, specialized neural networks.

Purpose of the Study:

• To develop a method for instantly estimating noise, MB, and DoF parameters from video.
• To enable the use of off-the-shelf, real-time simulation methods for AR content compositing.
• To achieve high-fidelity visual consistency between virtual and real content without manual calibration.

Main Methods:

• Utilizing modern computer vision techniques to remove noise, MB, and DoF from video streams.
• Implementing a self-calibration approach by leveraging these removal methods.
• Auto-tuning black-box real-time methods for noise, MB, and DoF.

Main Results:

• Instantaneous estimation of camera noise, MB, and DoF parameters.
• Successful integration of real-time simulation methods (e.g., game engines) for AR.
• Achieved fast and high-fidelity augmentation consistency.

Conclusions:

• The proposed method eliminates the need for traditional camera calibration in AR.
• It enables real-time, visually consistent AR experiences by effectively modeling camera distortions.
• This approach significantly advances the practicality and quality of AR applications.