TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Video Frame Interpolation (VFI) synthesizes intermediate frames between existing ones.
• Current VFI methods often fail with complex motions due to rigid motion pattern assumptions.
• This leads to misaligned features, causing distortion and blur in interpolated frames.

Purpose of the Study:

• To propose a novel Trajectory-aware Transformer for Video Frame Interpolation (TTVFI).
• To overcome limitations of pre-defined motion patterns in existing VFI techniques.
• To enhance the accuracy and quality of synthesized intermediate video frames.

Main Methods:

• Formulating inconsistently warped features as query tokens.
• Utilizing motion trajectory regions from consecutive frames as keys and values.
• Employing self-attention mechanisms along trajectories for feature blending in an end-to-end trained model.

Main Results:

• The proposed TTVFI method demonstrates superior performance compared to state-of-the-art approaches.
• Experiments were conducted on four widely-used VFI benchmarks.
• The method effectively handles complex and non-uniform motions, reducing distortion and blur.

Conclusions:

• TTVFI offers a significant advancement in video frame interpolation.
• The trajectory-aware approach effectively addresses challenges posed by complex motion patterns.
• The research provides a robust solution for generating high-quality intermediate video frames.