Wavelet-based joint estimation and encoding of depth-image-based representations for free-viewpoint rendering

Area of Science:

• Computer Vision
• Image Processing
• Signal Processing

Background:

• Depth-image-based representation (DIBR) enables novel view synthesis.
• Accurate depth map estimation is crucial for realistic DIBR.
• Existing methods often struggle with depth ambiguity and efficient encoding.

Purpose of the Study:

• To develop a novel wavelet-based codec for static DIBR.
• To improve the joint estimation and encoding of depth maps.
• To achieve rate-distortion (RD) optimized performance and scalability.

Main Methods:

• A novel RD optimization scheme is proposed for joint depth estimation and encoding.
• A rate constraint is used to favor piecewise-smooth depth maps, reducing estimation ambiguity.
• Dynamic programming along trees of integer wavelet coefficients efficiently solves the optimization.
• Joint encoding of images and depth maps minimizes redundancy and optimizes bitrate allocation.

Main Results:

• The proposed codec effectively estimates and encodes depth maps from multiple views.
• The RD optimization scheme leads to improved depth map quality and efficient bitrate usage.
• The codec demonstrates scalability in both resolution and quality.
• Experiments on real data validate the effectiveness of the proposed approach.

Conclusions:

• The wavelet-based codec offers a significant advancement for static DIBR.
• Joint optimization of depth estimation and encoding enhances visual quality and compression efficiency.
• The codec's scalability makes it suitable for diverse applications requiring free viewpoint selection.