A Deep Learning Approach for Multi-Frame In-Loop Filter of HEVC.

Area of Science:

• Computer Vision
• Digital Signal Processing
• Machine Learning

Background:

• Existing in-loop filters in High Efficiency Video Coding (HEVC) process frames individually, neglecting inter-frame correlations.
• This limitation leads to suboptimal artifact reduction and coding efficiency.

Purpose of the Study:

• To develop a multi-frame in-loop filter (MIF) for HEVC that enhances visual quality by utilizing information from adjacent frames.
• To improve coding efficiency beyond current HEVC standards.

Main Methods:

• Constructed a large-scale database of encoded and raw video frames.
• Developed a Reference Frame Selector (RFS) to identify high-quality, similar adjacent frames.
• Designed MIF-Net, a deep neural network based on DenseNet with a block-adaptive convolutional layer, to process spatial and temporal information.

Main Results:

• The proposed MIF approach achieved an average Bjøntegaard delta bit-rate (BD-BR) saving of 11.621% on standard test sets.
• Significantly outperformed the standard HEVC in-loop filter and other state-of-the-art methods in terms of coding efficiency.

Conclusions:

• The multi-frame in-loop filter (MIF) effectively enhances video quality and coding efficiency in HEVC.
• Leveraging temporal information from adjacent frames through deep learning offers a significant advancement over single-frame processing.