Deep Online Video Stabilization with Multi-Grid Warping Transformation Learning.

Area of Science:

• Computer Vision
• Deep Learning
• Video Processing

Background:

• Hand-held videos often suffer from high-frequency shakes, necessitating effective stabilization techniques.
• Existing video stabilization methods typically rely on offline algorithms and feature matching, which can be computationally intensive and fail on low-quality footage.
• Deep neural network-based solutions were previously unexplored due to data limitations and modeling challenges.

Purpose of the Study:

• To propose a novel deep neural network-based approach for real-time video stabilization.
• To address the limitations of existing offline methods, focusing on low-latency processing without future frame dependency.
• To develop a model capable of stabilizing challenging low-quality videos.

Main Methods:

• A convolutional neural network, named StabNet, was developed to learn mesh-grid transformations for progressive camera path smoothing.
• The model implicitly generates stable latent camera paths without explicit representation or reliance on future frames.
• A synchronized dataset of steady and unsteady video pairs was created using custom hardware for network training.

Main Results:

• StabNet achieves comparable performance to traditional offline video stabilization methods.
• The proposed online method operates approximately 10 times faster than existing techniques.
• StabNet demonstrates robustness in stabilizing low-quality videos, including night-scene, watermarked, blurry, and noisy footage, where conventional methods falter.

Conclusions:

• StabNet offers an efficient and effective solution for real-time video stabilization using deep learning.
• The network's ability to handle diverse low-quality video conditions represents a significant advancement over current methods.
• This work opens new avenues for deep learning applications in video processing and stabilization.