Prediction and Reference Quality Adaptation for Learned Video Compression.

Area of Science:

• Computer Science
• Signal Processing
• Artificial Intelligence

Background:

• Temporal prediction is crucial for video compression, with traditional codecs adapting coding modes based on prediction and reference quality.
• Learned video codecs show promise but struggle with effective prediction and reference quality adaptation, hindering performance and increasing error propagation.

Purpose of the Study:

• To enhance learned video codecs by addressing limitations in prediction and reference quality adaptation.
• To improve the utilization of temporal prediction and reduce reconstruction error propagation in video compression.

Main Methods:

• Proposed a confidence-based prediction quality adaptation (PQA) module to differentiate and adapt spatial and channel-wise prediction quality.
• Introduced a reference quality adaptation (RQA) module with a repeat-long training strategy for dynamic, spatially variant filters.
• Developed methods to suppress low-quality predictions and enhance high-quality ones, enabling adaptive selection of prediction locations.

Main Results:

• The PQA module allows codecs to adaptively select optimal spatial or channel locations for predictions based on quality.
• The RQA module and training strategy enable dynamic filtering for diverse reference qualities, improving reconstruction quality.
• Experimental results demonstrate significant improvements in compression performance using the proposed modules.

Conclusions:

• The developed PQA and RQA modules effectively enhance learned video codecs.
• These adaptations lead to better utilization of temporal prediction and reduced error propagation.
• The proposed methods achieve higher compression performance, advancing the field of learned video compression.