Patch-Wise Spatial-Temporal Quality Enhancement for HEVC Compressed Video.

Area of Science:

• Computer Vision
• Deep Learning
• Video Processing

Background:

• Current deep learning methods for compressed video enhancement often focus on spatial or temporal information independently.
• These approaches struggle to effectively combine spatial-temporal features, limiting performance on videos with scene changes or significant motion.

Purpose of the Study:

• To develop an advanced deep learning network for compressed video quality enhancement.
• To address the limitations of existing methods by adaptively integrating spatial and temporal information at a patch level.

Main Methods:

• A patch-wise spatial-temporal quality enhancement network was designed.
• The network employs a temporal and spatial-wise attention-based feature distillation structure to utilize adjacent patches.
• A channel and spatial-wise attention fusion block facilitates adaptive recalibration and fusion of features for each patch.

Main Results:

• The proposed network achieved a peak signal-to-noise ratio (PSNR) improvement of 0.55 - 0.69 dB over compressed videos.
• Performance gains were observed across various quantization parameters.
• The method outperformed existing state-of-the-art approaches.

Conclusions:

• The developed patch-wise spatial-temporal network effectively enhances compressed video quality.
• The adaptive fusion of spatial and temporal features is crucial for improving performance, particularly in complex video sequences.
• This research offers a significant advancement in deep learning-based video enhancement techniques.