Unsupervised Low-Light Video Enhancement With Spatial-Temporal Co-Attention Transformer

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Supervised Convolution Neural Networks (CNNs) dominate low-light video enhancement but struggle with real-world data due to synthetic training.
• Existing methods exhibit temporal inconsistency, like flickering and motion blur, especially with large motions, due to CNNs' limited perception of long-range dependencies.

Purpose of the Study:

• To develop the first unsupervised method for low-light video enhancement.
• To enhance video brightness while maintaining temporal consistency, even with large motions.
• To overcome the generalization limitations of supervised methods trained on synthetic data.

Main Methods:

• Proposed LightenFormer, an unsupervised low-light video enhancement method utilizing a spatial-temporal co-attention transformer.
• Introduced S-curve Estimation Network (SCENet) for adaptive dynamic range adjustment.
• Developed Spatial-Temporal Refinement Network (STRNet) with a novel Spatial-Temporal Co-attention Transformer (STCAT) for temporal consistency and long-range dependency modeling.
• Designed two non-reference loss functions for unsupervised training, leveraging S-curve invertibility and noise independence.

Main Results:

• LightenFormer effectively enhances brightness and maintains temporal consistency in low-light videos.
• The spatial-temporal co-attention transformer captures long-range spatial and temporal correlations for improved motion handling.
• Outperformed state-of-the-art methods on SDSD and LLIV-Phone datasets in extensive experiments.

Conclusions:

• LightenFormer offers a novel unsupervised approach to low-light video enhancement, addressing key limitations of prior supervised methods.
• The method demonstrates superior performance in brightness enhancement and temporal stability.
• Paves the way for more robust and generalizable low-light video enhancement techniques in real-world scenarios.