Temporal Fusion: Continuous-Time Light Field Video Factorization.

Area of Science:

• Computer Vision
• Display Technology
• 3D Graphics

Background:

• Factored displays offer glasses-free 3D by emitting full-parallax light fields without reducing spatial resolution.
• Static light field reconstruction uses frame-based low-rank factorization, but this causes artifacts and flickers in videos due to the continuous-time nature of perception.

Purpose of the Study:

• To develop a novel light-field video factorization framework to address artifacts and visual discomfort in glasses-free 3D displays.
• To incorporate the continuous-time persistence-of-vision (POV) effect into the factorization process for improved visual quality.

Main Methods:

• Introduced Temporal Fusion (TF), a framework that explicitly formulates the continuous-time POV effect into a global factorization objective.
• Developed sequence-level iterative updates on LCD sub-frames and an efficient cuboid-wise factorization algorithm for GPU implementation.
• Devised TF-C, a lightweight causal framework for low-latency applications.

Main Results:

• TF/TF-C significantly improves temporal consistency, reducing flicker values by 85%/91% compared to frame-based factorization.
• Reconstruction quality is enhanced, with PSNR values increasing by 5.0dB/3.7dB respectively.
• A prototype dual-layer factored display with high-refresh-rate LCDs demonstrated superior visual quality for real-life applications.

Conclusions:

• The Temporal Fusion framework effectively resolves visual flickers and enhances reconstruction quality for light field videos.
• The proposed methods enable practical GPU implementation and support low-latency applications, paving the way for improved glasses-free 3D experiences.
• Experimental results and prototype demonstration validate the effectiveness of TF/TF-C in real-world scenarios.