DUT: Learning Video Stabilization by Simply Watching Unstable Videos

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Deep learning video stabilizers require extensive paired data.
• Traditional methods struggle with textureless/occluded regions due to hand-crafted features.

Purpose of the Study:

• To develop a deep unsupervised learning method for video stabilization.
• To address limitations of existing deep learning and traditional approaches.

Main Methods:

• Introduced DUT (Deep Unsupervised learning Trajectory-based) video stabilizer.
• Employs a divide-and-conquer strategy with deep neural networks (DNNs).
• Comprises trajectory estimation (keypoint motion, multi-homography, motion refinement) and trajectory smoothing (dynamic smoothing kernels).

Main Results:

• Achieved state-of-the-art performance on public benchmarks.
• Demonstrated superior qualitative and quantitative results compared to existing methods.
• Successfully trained in an unsupervised manner by exploiting spatial and temporal coherence.

Conclusions:

• The proposed deep unsupervised method offers a robust solution for video stabilization.
• DUT effectively handles challenges in real-world scenarios without paired training data.
• The method shows significant improvements over current state-of-the-art techniques.