Temporally consistent low-light face video enhancement via video-to-video conditional diffusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Low-light face videos exhibit significant noise and loss of detail.
• This degradation limits their utility in critical applications like surveillance and photography.

Purpose of the Study:

• To introduce DL-Diff, a novel framework for low-light face video enhancement.
• To address noise and detail loss in low-light videos using conditional video-to-video generation.

Main Methods:

• DL-Diff leverages pre-trained Latent Diffusion Models (LDMs) for video-to-video generation.
• The framework incorporates a pseudo-3D UNet backbone, spatial detail restoration, and inter-frame temporal consistency components.
• A multi-stage training strategy facilitates domain adaptation from image to video enhancement.

Main Results:

• DL-Diff achieved superior performance in perceptual quality (FID: 41.29, LPIPS: 0.17) and temporal consistency (AB(Var): 25.40, MABD: 0.08).
• The framework significantly outperformed existing low-light video enhancement methods.
• Generated videos exhibited realistic visual effects, free from flickering artifacts, especially in extreme darkness.

Conclusions:

• DL-Diff effectively enhances low-light face videos, improving both visual quality and temporal coherence.
• The study demonstrates the efficacy of pre-trained diffusion models for advanced video enhancement tasks.
• This framework holds promise for improving surveillance and photographic applications under challenging lighting conditions.