An Improved Augmented-Reality Framework for Differential Rendering Beyond the Lambertian-World Assumption.

Area of Science:

• Computer Vision
• Computer Graphics
• Human-Computer Interaction

Background:

• Achieving visual consistency between virtual objects and real scenes is crucial in augmented reality (AR).
• Specular and transparent objects can create caustics, negatively impacting the visual integration of virtual elements.
• Existing methods often rely on simplified lighting assumptions (Lambertian-world), limiting realism.

Purpose of the Study:

• To propose a novel framework for differential rendering that goes beyond the Lambertian-world assumption.
• To enhance the visual consistency of augmented reality by accurately rendering specular and transparent objects.
• To jointly optimize scene illumination and object parameters for realistic AR integration.

Main Methods:

• Developed a differential rendering framework to handle complex optical phenomena like caustics.
• Introduced a joint optimization approach for illumination and parameters of specular and transparent objects.
• Incorporated a psychophysical scaling method, considering human visual characteristics, to efficiently estimate the refractive index of transparent objects.

Main Results:

• The proposed framework successfully renders specular and transparent objects with improved visual fidelity.
• Experimental results on multiple real scenes demonstrate visually consistent fusion of virtual objects.
• The psychophysical scaling method provides an efficient way to estimate refractive indices, enhancing parameter estimation.

Conclusions:

• The developed framework effectively addresses the challenge of visual consistency in augmented reality for complex object types.
• Jointly optimizing illumination and object parameters, coupled with human-centric estimation, leads to superior rendering results.
• This approach advances the realism and believability of augmented reality experiences.