Deep Learning Post-Filtering Using Multi-Head Attention and Multiresolution Feature Fusion for Image and Intra-Video...

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Lossy compression using codecs like JPEG, JPEG2000, HEVC, and VVC degrades image and video quality.
• Existing post-filtering methods struggle to recover fine details and achieve significant quality enhancement.

Purpose of the Study:

• To propose a novel post-filtering method for quality enhancement of compressed RGB/grayscale images and video sequences.
• To develop a deep neural network architecture capable of estimating fine refinement details at multiple resolutions.

Main Methods:

• A novel deep neural network architecture using efficient processing blocks.
• Incorporation of multi-head attention for feature map refinement.
• Utilization of weight sharing and novel block designs for multiresolution feature fusion.

Main Results:

• Substantial performance improvements over common image codecs (JPEG, JPEG2000) and video coding standards (HEVC, VVC).
• Average BD-rate savings of 31.44% (RGB) and 26.21% (grayscale) over JPEG.
• Significant BD-rate savings over HEVC (54.61% RGB) and VVC (15.28% grayscale).
• Average BD-rate savings of 15.47% (HEVC) and 14.66% (VVC) for video sequences.

Conclusions:

• The proposed CNN-based post-filtering method effectively enhances the quality of compressed images and videos.
• The novel network architecture demonstrates superior performance in recovering lost details and reducing bitrate.
• This approach offers a promising solution for improving visual quality in digital media.