Leveraging Temporal Down-Sampling Structure and Spatio-Temporal Fusion for Efficient Video Coding

Area of Science:

• Computer Vision
• Image and Video Processing
• Data Compression

Background:

• Modern video compression relies on down-sampling, but current methods overlook spatial-temporal redundancy, leading to information loss.
• Uniform frame down-sampling in existing systems degrades compression efficiency and video quality.

Purpose of the Study:

• To propose an advanced temporal down-sampling video compression system that addresses limitations of current methods.
• To enhance compression efficiency by preserving key frames and intelligently down-sampling intermediate frames.

Main Methods:

• A novel temporal down-sampling strategy is introduced, selectively down-sampling intermediate frames while retaining high-quality key frames.
• A frame-recurrent enhancement mechanism is employed at the decoder to leverage temporal redundancy.
• A Multi-scale Temporal-Spatial Attention (MTSA) module, comprising Multi-Temporal Attention (MTA) and Pyramid Spatial Attention (PSA), is designed for the enhancement stage.

Main Results:

• The proposed system achieves consistent BD-rate reductions across various configurations (All-Intra, Low-Delay-P, Random Access).
• Significant BD-rate reductions of 14% to 39% for I, P, and B frames were observed compared to the Versatile Video Coding (VVC) standard.
• The approach demonstrates superior performance over existing methods, including those based on the High Efficiency Video Coding (HEVC) standard.

Conclusions:

• The proposed temporal down-sampling system effectively utilizes spatial and temporal redundancy for improved video compression.
• The MTSA module plays a crucial role in enhancing frame quality by modeling temporal correlations and spatial saliency.
• This method offers a promising direction for next-generation video compression, outperforming current state-of-the-art standards.