Generative Adversarial Networks and Perceptual Losses for Video Super-Resolution.

Area of Science:

• Computer Vision
• Deep Learning
• Video Processing

Background:

• Video super-resolution (VSR) is a critical challenge in video processing.
• Adversarial and perceptual losses have improved image restoration but are underutilized in VSR.
• Existing VSR methods lack robust evaluation metrics for perceptual quality.

Purpose of the Study:

• To propose a novel Generative Adversarial Network (GAN)-based framework for video super-resolution (VSR).
• To introduce a new generator network (VSRResNet) and discriminator architecture tailored for VSR.
• To enhance the GAN formulation with feature-space and pixel-space regularizers for superior performance.

Main Methods:

• Developed a new generator network, VSRResNet, optimized for VSR tasks.
• Designed a novel discriminator architecture to guide GAN training for VSR.
• Incorporated feature-space and pixel-space distance losses as regularizers, creating the VSRResFeatGAN model.
• Utilized the PercepDist metric for evaluating perceptual quality, alongside traditional metrics.

Main Results:

• Pre-training the generator with mean-squared-error loss alone surpassed current state-of-the-art VSR models quantitatively.
• The proposed VSRResFeatGAN model demonstrated superior performance compared to existing VSR methods.
• The PercepDist metric proved more effective in evaluating the perceptual quality of super-resolved videos than PSNR/SSIM.

Conclusions:

• The proposed VSRResFeatGAN model represents a significant advancement in video super-resolution.
• Adversarial learning with specialized network architectures and regularizers effectively enhances VSR performance.
• The PercepDist metric offers a more reliable evaluation of perceptual quality in deep learning-based super-resolution.