Learning Bilateral Cost Volume for Rolling Shutter Temporal Super-Resolution.

Area of Science:

• Computer Vision
• Deep Learning
• Video Processing

Background:

• Rolling shutter (RS) cameras capture images sequentially, leading to distortions.
• Existing rolling shutter temporal super-resolution (RSSR) methods use complex, multi-stage networks.
• Current methods struggle with pixel aliasing and integrating motion/appearance information.

Purpose of the Study:

• To develop a unified, single-stage RSSR pipeline for synthesizing intermediate global shutter (GS) frames.
• To improve motion field estimation and frame synthesis accuracy.
• To overcome limitations of existing cascaded network approaches.

Main Methods:

• Derivation of uniform bilateral motion fields for RS-aware backward warping.
• Development of a unified, single-stage RSSR pipeline using pyramid features and coarse-to-fine refinement.
• Introduction of a bilateral cost volume for modeling correlations between RS frames and an anchor frame.
• Implementation of an asymmetric bilateral motion model on top of a symmetric one.

Main Results:

• Achieved new state-of-the-art performance in quantitative and qualitative experiments.
• Demonstrated improved motion estimation and contextual detail compensation.
• Eliminated the need for additional frame synthesis or refinement networks.

Conclusions:

• The proposed unified, single-stage RSSR method offers a more effective and efficient solution for GS video reconstruction.
• The novel bilateral motion fields and cost volume significantly enhance spatio-temporal consistency and accuracy.
• The method shows strong generality and adaptability on both synthetic and real-world data.