STDAN: Deformable Attention Network for Space-Time Video Super-Resolution.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Video Processing

Background:

• Current space-time video super-resolution (STVSR) methods often rely on limited temporal information (adjacent frames).
• Existing models inadequately leverage temporal contexts for high-resolution (HR) frame reconstruction.
• This limits the full exploration of information flow in consecutive low-resolution (LR) video frames.

Purpose of the Study:

• To propose a novel deformable attention network, STDAN, for improved STVSR.
• To address the limitations of short-term feature utilization in current STVSR approaches.
• To enhance the exploitation of temporal contexts for superior HR video reconstruction.

Main Methods:

• Developed a long short-term feature interpolation (LSTFI) module using bidirectional recurrent neural networks (RNNs).
• Introduced a spatial-temporal deformable feature aggregation (STDFA) module for adaptive context capture.
• Employed a deformable attention mechanism within the STDAN framework.

Main Results:

• STDAN effectively utilizes information from multiple neighboring frames for interpolation.
• The STDFA module adaptively captures and aggregates spatial-temporal contexts.
• Experimental results show STDAN surpasses state-of-the-art STVSR methods on benchmark datasets.

Conclusions:

• The proposed STDAN model significantly advances STVSR performance.
• Leveraging long-term and spatial-temporal contexts is crucial for effective video super-resolution.
• The developed LSTFI and STDFA modules offer a robust approach to enhance video quality.