Multi-Scale Spatio-Temporal Memory Network for Lightweight Video Denoising.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Signal Processing

Background:

• Deep learning significantly advanced video denoising but suffers from high computational demands.
• Existing methods struggle with real-world applications due to computational complexity.

Purpose of the Study:

• To develop a fast video denoising algorithm with an improved cost-performance trade-off.
• Introduce a lightweight network, the Multiscale Spatio-Temporal Memory Network (MSTMN), for efficient video denoising.

Main Methods:

• Utilized a multiscale representation via Gaussian-Laplacian pyramid decomposition for coarse-to-fine restoration.
• Incorporated variance estimation, alignment error estimation, and adaptive fusion modules guided by model-based optimization.
• Employed a reconstruction recurrence strategy and a memory enhancement module for temporal and global spatio-temporal information integration.
• Leveraged patch-level similarity computation to avoid complex motion estimation.

Main Results:

• The proposed MSTMN network demonstrated superior performance compared to state-of-the-art fast video denoising algorithms.
• Achieved significant denoising improvements with substantially lower computational costs on real-world raw video datasets.
• Outperformed methods like FastDVDnet, EMVD, and ReMoNet.

Conclusions:

• MSTMN offers an effective solution for fast video denoising, balancing high performance with efficiency.
• The lightweight design and adaptive information search make it suitable for practical, real-world applications.
• This approach advances the field by reducing computational barriers in deep learning-based video denoising.