Hybrid Compression Method for Trained 3D Gaussian Splatting Models Based on VQ and HEVC

Area of Science:

• Computer Graphics and Vision
• Image and Video Compression
• Machine Learning for Media

Background:

• 3D Gaussian Splatting (3DGS) offers high-fidelity, real-time 3D scene rendering.
• Gaussian Splat Coding (GSC) is being explored by MPEG for interoperable 3DGS content compression.
• Current GSC methods using HEVC struggle with coding efficiency for spherical harmonic coefficients (SH-AC) due to their irregular nature.

Purpose of the Study:

• To propose a novel VQ-HEVC hybrid compression framework for 3DGS content.
• To enhance the coding efficiency of SH-AC within the GSC framework.
• To maintain structural compatibility with the existing video-based GSC anchor.

Main Methods:

• Representing SH-AC coefficients using vector quantization (VQ) indices.
• Employing a two-stage VQ scheme: coarse VQ and product-quantization-based residual quantization.
• Encoding VQ indices via HEVC lossless coding on YUV400 index-map sequences, with codebooks transmitted as metadata.

Main Results:

• Consistent rate-distortion improvements over the video-based GSC anchor were demonstrated.
• Achieved BD-rate reductions of 22.3% (Bartender) and 18.5% (Cinema) in RGB-PSNR.
• The VQ-based SH-AC representation effectively complements PLAS-based video coding.

Conclusions:

• The proposed VQ-HEVC hybrid framework significantly improves 3DGS compression efficiency.
• VQ-based SH-AC representation is a viable enhancement for GSC, maintaining structural consistency.
• This method offers a promising direction for efficient compression of advanced 3D representations.