Exploiting Latent Properties to Optimize Neural Codecs

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• End-to-end neural codecs offer advantages over traditional methods, including adaptability to perceptual metrics and high performance.
• Current neural codecs underutilize vector quantization and the decoder-available entropy gradient.

Purpose of the Study:

• To enhance the performance of existing image and video codecs by leveraging vector quantization and entropy gradient properties.
• To investigate the effectiveness of uniform vector quantization over non-uniform scalar quantization.
• To utilize the entropy gradient as a proxy for the reconstruction error gradient to improve compression.

Main Methods:

• Investigated the impact of uniform versus non-uniform scalar quantization, proposing optimal uniform vector quantization.
• Exploited the entropy gradient at the decoder as a proxy for the unavailable reconstruction error gradient.
• Evaluated performance improvements across various pre-trained neural codec models.

Main Results:

• Optimal uniform vector quantization improves performance over non-uniform scalar quantization.
• The entropy gradient effectively serves as a proxy for the reconstruction error gradient, enhancing compression.
• Achieved 1-3% bitrate savings for the same quality across tested neural codecs.
• Demonstrated significant performance improvements for traditional codecs using the entropy gradient method.

Conclusions:

• Leveraging optimal uniform vector quantization and the entropy gradient offers a significant performance boost for neural codecs.
• The proposed methods are effective in improving compression efficiency for both modern neural and traditional codecs.
• The entropy gradient provides a viable and effective mechanism for enhancing compression performance at the decoder.