Factored Occlusion: Single Spatial Light Modulator Occlusion-capable Optical See-through Augmented Reality Display.

Area of Science:

• Computer Vision
• Optics
• Human-Computer Interaction

Background:

• Occlusion is vital for depth perception and realism in optical see-through augmented reality (OST-AR).
• Current OST-AR systems struggle with realistic mutual occlusion due to additive light combination, causing semi-transparent virtual objects.
• Existing methods lack the ability to achieve hard-edge occlusion between real and virtual elements.

Purpose of the Study:

• To propose and demonstrate a novel occlusion-capable OST-AR system.
• To overcome the limitations of additive light combination in current OST-AR displays.
• To achieve realistic mutual occlusion, including hard-edge occlusions and shadows, in OST-AR.

Main Methods:

• Developed a multiplicative light path merging technique using a single digital micromirror device (DMD).
• Employed mixed binary/continuous factorization algorithms to optimize time-multiplexed binary DMD patterns and LED colors.
• Validated the approach through simulations and a prototype benchtop display.

Main Results:

• Demonstrated hard-edge occlusions between real and virtual objects.
• Achieved plausible shadow generation for enhanced realism.
• Showcased gaze-contingent optimization of the novel display mode.

Conclusions:

• The proposed multiplicative approach using a single DMD enables realistic mutual occlusion in OST-AR.
• This technique significantly improves the visual fidelity and immersion of augmented reality experiences.
• The system offers a promising direction for future occlusion-capable OST-AR displays.