Affine Transformation-Based Deep Frame Prediction.

Area of Science:

• Computer Vision
• Machine Learning
• Video Compression

Background:

• Traditional video coding relies heavily on motion estimation, which requires transmitting motion information.
• Existing deep learning models for frame prediction can be computationally intensive and require complex filter designs.

Purpose of the Study:

• To develop a novel neural network model for accurate and efficient deep frame prediction.
• To reduce the bit rate in video coding by eliminating the need to transmit motion information.

Main Methods:

• Proposed a neural network model utilizing affine transformation and adaptive spatially-varying filters for frame estimation.
• Employed dilated convolutions and reduced filter lengths for a more compact and efficient model.
• Trained two model versions (uni-directional and bi-directional) using a combination of DCT-based l1-loss, multi-scale MSE loss, and object context reconstruction loss.

Main Results:

• The proposed model achieves significant bit savings in the HEVC (High Efficiency Video Coding) pipeline.
• Achieved average bit savings of 7.3% (Low delay P), 5.4% (Low delay), and 4.2% (Random access) for the luminance component.
• Demonstrated superior accuracy and reduced model size compared to existing neural network approaches for deep frame prediction.

Conclusions:

• The developed neural network model offers an efficient and accurate solution for deep frame prediction in video coding.
• Eliminating motion information transmission through effective frame prediction leads to substantial bit rate reduction.
• The model's integration into HEVC shows practical applicability and performance improvements in various coding configurations.