Moment-Reenacting: Inverse Motion Degradation With Cross-Shutter Guidance

Area of Science:

• Computational Imaging
• Computer Vision
• Image Processing

Background:

• Motion degradation, including global shutter (GS) blur and rolling shutter (RS) distortion, poses significant challenges in capturing clear images under fast motion or low light.
• Existing methods often address blur decomposition and RS temporal super-resolution independently, neglecting their inherent complementarity.

Purpose of the Study:

• To propose a unified framework for inverting motion degradation and reconstructing imaging moments by leveraging the complementary information from GS and RS imaging.
• To resolve temporal and spatial ambiguities present in both GS and RS modalities.

Main Methods:

• Introduced a novel dual-shutter setup capturing synchronized GS-RS image pairs.
• Extended the dual-shutter setup to a stereo Blur-RS configuration for performance-cost trade-offs.
• Developed a triaxial imaging system for a real-world dataset with aligned GS-RS pairs and ground-truth high-speed frames.
• Proposed a network with a dual-stream motion interpretation module and a self-prompted frame reconstruction stage.

Main Results:

• The dual-shutter approach effectively resolves temporal and spatial ambiguities.
• The proposed network successfully disentangles motion into context-aware and temporally-sensitive representations.
• Extensive experiments demonstrate the superiority and generalizability of the approach over existing methods.

Conclusions:

• The unified framework establishes a new paradigm for realistic high-speed video reconstruction under complex motion degradations.
• Jointly leveraging GS and RS data offers significant advantages over separate processing.
• The developed system and dataset enable robust training and evaluation for motion-degraded imaging.