Graph Neural Network-Based GrUNet and Attention Transformer Adjacency Matrix for Video Denoising

Videos account for a significant portion of internet traffic, and the presence of noise, whether from compression algorithms, low light, sensor imperfections, deteriorates the video quality. Ambient noise can also significantly diminish the visual quality. Traditional CNN-based video denoising methods rely on convolutional filters with fixed sizes and receptive fields, excelling at capturing local patterns and short-range dependencies. However, CNNs often struggle to handle long-term dependencies or relationships that extend over larger spatial and temporal scales. These are vital for accurately removing noise while preserving essential video details, textures, and structures. To address this limitation, we propose a novel approach, using UNet architecture, which combines the strengths of convolutional neural networks (CNNs) and graph neural networks (GNNs) for local and global information and dependency preservation. In this approach, CNN is followed by transformer attention for sparse graph formation for CNN. The spatiotemporal patches act as nodes, and the similarity between them represent edges. By integrating CNNs for local feature extraction followed by transformer attention and GNN for video denoising first time, for long-term spatio-temporal relationships, improves the ability to accurately model noise, preserve fine details and subsequently denoise videos more accurately. The strong ablation studies prove the effectiveness of the different modules, patch sizes on four different noise types. The proposed method outperformed most of the SOTA video denoising algorithms in terms of both PSNR and SSIM, at moderate computational cost, apart from the Video Restoration Transformer(VRT).