Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression.

Area of Science:

• Computer Vision
• Machine Learning
• Video Compression

Background:

• Neural video codecs show promise for video transmission and storage.
• Current methods lack fine-grained adaptation to diverse motion content due to reliance on basic flow prediction.

Purpose of the Study:

• To develop a more content-adaptive neural video codec.
• To improve motion compensation and rate-distortion performance.

Main Methods:

• Proposed a novel cross-scale prediction module utilizing a reference feature pyramid and cross-scale flows.
• Introduced a weighted prediction mechanism using cross-scale weight maps with a single reference frame.
• Implemented a multi-stage quantization strategy for enhanced rate-distortion performance.

Main Results:

• The efficient neural video codec (ENVC) demonstrates encouraging performance on benchmark datasets.
• ENVC competes with H.266/VVC in sRGB PSNR on the UVG dataset for low-latency mode.
• The cross-scale prediction module effectively handles various video content.

Conclusions:

• The proposed cross-scale prediction module and multi-stage quantization significantly improve neural video codec efficiency.
• ENVC offers a competitive alternative to existing video coding standards, particularly in low-latency scenarios.