A Temporally-Aware Interpolation Network for Video Frame Inpainting

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Video frame inpainting is crucial for reconstructing missing video content.
• Existing methods often fail to utilize the full contextual information available in video sequences.
• This limitation impacts the quality of inpainting, interpolation, and prediction tasks.

Purpose of the Study:

• To develop a specialized method for video frame inpainting.
• To effectively utilize preceding and following frames as context for predicting missing frames.
• To improve accuracy and smoothness in video reconstruction tasks.

Main Methods:

• A novel two-module approach: bidirectional video prediction and temporally-aware frame interpolation.
• Utilizing a convolutional LSTM-based encoder-decoder for intermediate frame predictions.
• Blending predictions using temporal information and hidden activations to resolve discrepancies.

Main Results:

• The proposed method achieves superior performance compared to state-of-the-art techniques.
• Demonstrated improvements in smoothness and accuracy for video frame inpainting.
• Outperformed general video inpainting, frame interpolation, and video prediction methods.

Conclusions:

• The developed method effectively addresses video frame inpainting by leveraging contextual information.
• The combination of prediction and interpolation modules enhances reconstruction quality.
• This approach sets a new benchmark for video frame inpainting and related tasks.