Online Streaming Video Super-Resolution With Convolutional Look-Up Table.

Area of Science:

• Computer Vision
• Video Processing
• Machine Learning

Background:

• Online video streaming faces bandwidth and computational limits, hindering high-quality video delivery.
• Existing super-resolution (SR) methods are ill-suited for dynamic degradations and strict latency requirements of real-time streaming protocols like WebRTC.
• The problem of super-resolution for online streaming video remains largely unexplored.

Purpose of the Study:

• To address the challenges of super-resolution in online video streaming.
• To develop a novel method optimized for the unique constraints of real-time video transmission.
• To establish a benchmark dataset for evaluating online video super-resolution techniques.

Main Methods:

• A new benchmark dataset, LDV-WebRTC, was created using a real-world online streaming system.
• A novel convolution and Look-Up Table (LUT) hybrid model was proposed for an effective performance-latency trade-off.
• A mixture-of-expert-LUT module was developed to adaptively handle diverse and dynamic video degradations.

Main Results:

• The proposed method achieves 720P video super-resolution at approximately 100 FPS.
• The method significantly outperforms existing LUT-based super-resolution techniques.
• It offers competitive performance compared to efficient Convolutional Neural Network (CNN)-based methods.

Conclusions:

• The developed method effectively tackles the challenges of online streaming video super-resolution.
• The proposed approach provides a superior performance-latency trade-off for real-time applications.
• The LDV-WebRTC dataset and the novel method facilitate further research in this domain.