JNMR: Joint Non-Linear Motion Regression for Video Frame Interpolation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Signal Processing

Background:

• Video frame interpolation (VFI) generates intermediate frames using motion-warping from reference frames.
• Current VFI methods struggle with non-linear motion due to variable acceleration and irregular trajectories.
• Spatiotemporal semantic information is commonly used but insufficient for complex real-world scenarios.

Purpose of the Study:

• To address limitations in VFI for non-linear motion estimation.
• To propose a new strategy for modeling complicated inter-frame motions.
• To enhance the accuracy and quality of interpolated video frames.

Main Methods:

• Reformulated VFI as a joint non-linear motion regression (JNMR) strategy.
• Regressed motion trajectories using multi-stage quadratic models and a joint distribution.
• Employed a feature learning network with dense skip-connections for contextual details.
• Utilized a coarse-to-fine synthesis enhancement module for multi-scale texture learning.

Main Results:

• Demonstrated significant improvements in VFI performance compared to state-of-the-art methods.
• Showcased the effectiveness of the JNMR strategy in handling complex inter-frame motions.
• Achieved superior visual quality and accuracy in generated interpolated frames.

Conclusions:

• The proposed JNMR strategy offers a robust solution for challenging video frame interpolation tasks.
• This approach effectively captures non-linear motion dynamics, outperforming existing techniques.
• The method provides a significant advancement in video synthesis and processing.