Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Video quality is limited by sensor capabilities, necessitating video super-resolution (VSR) technologies.
• Developing dedicated VSR models is computationally expensive and resource-intensive.

Purpose of the Study:

• To propose a novel and cost-effective method for adapting existing single-image super-resolution (SISR) models for VSR tasks.
• To enhance the performance of SISR models for video enhancement without requiring entirely new architectures.

Main Methods:

• A formal analysis of SISR model adaptation was performed.
• A plug-and-play temporal feature extraction module was developed, comprising offset estimation, spatial aggregation, and temporal aggregation submodules.
• The module aligns and fuses features from multiple frames before feeding them into the SISR model for reconstruction.

Main Results:

• Five representative SISR models were successfully adapted using the proposed method.
• Adapted models showed significant improvements on the Vid4 benchmark, with PSNR increasing by at least 1.26 dB and SSIM by 0.067.
• The VSR-adapted models outperformed current state-of-the-art VSR methods.

Conclusions:

• The proposed adaptation method is effective across various SISR models for improving video super-resolution.
• This approach offers a practical and efficient solution for enhancing video quality, reducing the cost associated with VSR model development.