Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Video.

Area of Science:

• Computer Vision
• Image Processing
• Deep Learning

Background:

• Rolling-shutter (RS) cameras are common but introduce motion distortion.
• This distortion hinders performance in computer vision tasks.
• Existing methods lack practical solutions for RS distortion correction.

Purpose of the Study:

• To develop a deep learning framework for rolling-shutter temporal super-resolution (RSSR).
• To recover continuous high-framerate global-shutter (GS) video from two RS frames.
• To address the challenge of significant image distortion caused by RS cameras.

Main Methods:

• Proposed a novel deep-learning framework leveraging multi-view geometry.
• Introduced bidirectional RS undistortion flow formulations for constant velocity and acceleration.
• Developed a linear scaling operation to bridge RS undistortion flow and optical flow.
• Implemented a mutual conversion scheme for RS undistortion flows across scanlines.
• Exploited spatial-temporal geometric relationships without extensive supervision.

Main Results:

• Successfully generated high-framerate GS video from two RS frames.
• Achieved high-quality GS image sequences with rich details.
• Demonstrated superior performance over state-of-the-art methods on synthetic and real data.
• Presented the first deep network capable of rolling-shutter temporal super-resolution using only two frames.

Conclusions:

• The proposed method effectively inverts the rolling-shutter capture mechanism.
• The framework provides a practical solution for RS temporal super-resolution.
• The approach significantly enhances the quality of GS video recovery from RS images.