Blind Quality Prediction for View Synthesis Based on Heterogeneous Distortion Perception

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Virtual view synthesis quality is crucial for practical applications.
• Current quality metrics rely on computationally expensive Depth-Image-Based Rendering (DIBR) and lack robustness.
• Existing metrics use shallow, hand-crafted features, limiting their effectiveness.

Purpose of the Study:

• To develop a computationally efficient blind quality prediction model for view synthesis.
• To avoid the complex DIBR process by directly predicting quality from input images.
• To learn robust and efficient features for synthesized image quality assessment.

Main Methods:

• A novel blind quality prediction model, HEDIP (HEterogeneous DIstortion Perception), is proposed.
• Texture and depth images are fused using Discrete Cosine Transform (DCT) to simulate distortions.
• A Two-Channel Convolutional Neural Network (TCCNN) extracts spatial and gradient features, enhanced by a Heterogeneous Distortion Perception (HDP) module for training labels.

Main Results:

• The HEDIP model effectively predicts the quality of synthesized images.
• The Heterogeneous Distortion Perception (HDP) module provides effective training labels, addressing local distortions.
• Experimental results demonstrate the model's effectiveness and robustness in quality prediction.

Conclusions:

• HEDIP offers an efficient alternative to DIBR-based quality assessment for virtual view synthesis.
• The integration of DCT fusion and TCCNN with the HDP module significantly improves prediction accuracy.
• The proposed model advances the field of blind image quality assessment for synthesized views.