QP-Adaptive Dual-Path Residual Integrated Frequency Transformer for Data-Driven In-Loop Filter in VVC.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Signal Processing

Background:

• AI-enabled embedded systems require efficient video processing, making Versatile Video Coding (VVC/H.266) crucial for Multimedia Internet of Things (M-IoT).
• VVC's block-based coding can cause compression artifacts, and existing Convolutional Neural Network (CNN) methods struggle with performance across different Quantization Parameters (QPs).

Purpose of the Study:

• To propose DRIFT, a novel QP-adaptive in-loop filtering network designed to reduce compression artifacts in VVC.
• To enhance video processing efficiency and quality for AI-enabled systems and M-IoT applications.

Main Methods:

• DRIFT integrates a lightweight frequency fusion CNN (LFFCNN) for local enhancement and a Swin Transformer for global context.
• LFFCNN utilizes octave convolution and a novel residual block (FFRB) incorporating multiscale extraction, QP adaptivity, frequency fusion, and attention mechanisms.
• A Quantization Parameter Estimator (QPE) is introduced to prevent over-enhancement in inter-coded frames.

Main Results:

• DRIFT achieved significant BD rate reductions: 6.56% for intra-coded frames and 4.83% for inter-coded frames.
• The BasketballDrill sequence showed up to 10.90% performance gain.
• LFFCNN reduced model size by 32% compared to prior methods while maintaining or improving coding performance.

Conclusions:

• DRIFT effectively addresses VVC compression artifacts, offering robust performance across various QPs.
• The proposed LFFCNN and QPE contribute to efficient and high-quality video processing for modern AI applications.
• DRIFT represents a significant advancement in QP-adaptive filtering for VVC, outperforming existing solutions.