Deep end-to-end rolling shutter rectification.

Area of Science:

• Computer Vision
• Image Processing
• Deep Learning

Background:

• CMOS sensors capture images row-by-row, leading to rolling shutter (RS) distortions.
• Current RS rectification methods require specific scene information or known motion parameters.

Purpose of the Study:

• To develop an end-to-end deep neural network for single image rolling shutter rectification.
• To address limitations of existing methods by handling complex, real-life camera motion without ground truth parameters.

Main Methods:

• An end-to-end deep neural network comprising motion, trajectory, row, and rectification modules.
• The network predicts row-wise camera poses and fits them to a polynomial trajectory.
• A regeneration module aids training by comparing input and ground truth distorted images.

Main Results:

• The proposed network successfully rectifies rolling shutter distortions in single images.
• It effectively handles complex camera motion without needing ground truth motion parameters.
• Experiments show superior qualitative and quantitative performance compared to prior art on synthetic and real datasets.

Conclusions:

• The developed deep neural network offers a robust solution for single image rolling shutter rectification.
• Its end-to-end formulation and ability to handle complex motion represent a significant advancement in image processing.
• The method demonstrates strong potential for real-world applications requiring distortion-free images.