Deep Video Prior for Video Consistency and Propagation

Area of Science:

• Computer Vision
• Video Processing
• Machine Learning

Background:

• Image processing algorithms applied frame-by-frame often cause temporal inconsistencies in videos.
• Existing methods typically rely on large datasets or optical flow for temporal consistency, limiting their generalizability.

Purpose of the Study:

• To develop a novel and general approach for blind video temporal consistency.
• To address multimodal inconsistency problems in video processing.
• To extend Deep Video Prior (DVP) for effective video information propagation.

Main Methods:

• A convolutional neural network trained on a single pair of original and processed videos using Deep Video Prior (DVP).
• An iteratively reweighted training strategy to handle multimodal inconsistency.
• Extension of DVP for propagating color, artistic style, and object segmentation information.

Main Results:

• Achieved superior performance in blind video temporal consistency compared to state-of-the-art methods across 7 computer vision tasks.
• Demonstrated the effectiveness of DVP in propagating various types of information in videos.
• Showcased the method's ability to train on minimal data (one video pair).

Conclusions:

• Deep Video Prior (DVP) offers an effective solution for blind video temporal consistency without large datasets.
• DVP can be successfully extended for robust video information propagation tasks.
• The proposed methods significantly advance the state-of-the-art in video processing and consistency.