Insights From Generative Modeling for Neural Video Compression.

Area of Science:

• Machine Learning
• Computer Vision
• Signal Processing

Background:

• Recent machine learning research connects deep generative models (e.g., VAEs) with rate-distortion losses in learned compression, primarily for images.
• Neural video coding algorithms are emerging as a new frontier in video compression technology.

Purpose of the Study:

• To analyze neural video coding algorithms through the framework of deep autoregressive and latent variable modeling.
• To propose novel architectures and improvements for state-of-the-art video compression based on generative modeling principles.

Main Methods:

• Viewing neural video codecs as generalized stochastic temporal autoregressive transforms.
• Developing new architectures inspired by normalizing flows and structured priors.
• Implementing improved temporal autoregressive transforms and entropy models with structured/temporal dependencies.

Main Results:

• Achieved state-of-the-art video compression performance on high-resolution video datasets.
• Demonstrated the effectiveness of proposed architectures through detailed tradeoffs and ablation studies.
• Introduced variable bitrate versions of the algorithms, enhancing flexibility.

Conclusions:

• The generative modeling viewpoint offers significant potential for advancing the field of neural video coding.
• Proposed improvements are compatible with a wide range of existing neural video coding models.
• This work provides a strong foundation for future research in AI-driven video compression.