HoloQA: Full Reference Video Quality Assessor of Rendered Human Avatars in Virtual Reality.

Area of Science:

• Computer Vision
• Virtual Reality
• Digital Signal Processing

Background:

• Virtual Reality (VR) and Augmented Reality (AR) systems increasingly use digital human avatars.
• Transmitting these avatars over networks requires robust video quality assessment (VQA) to address unique distortions.
• Existing VQA models often fail to capture avatar-specific rendering, transmission, and compression artifacts.

Purpose of the Study:

• To develop a state-of-the-art Full Reference Video Quality Assessment (VQA) model for digital human avatars in VR/AR.
• To address the limitations of standard VQA models in evaluating avatar quality.
• To provide a VQA solution that accounts for distortions specific to rendered and transmitted human avatars.

Main Methods:

• Developed HoloQA, a VQA model based on visual neuroscience, information theory, and self-supervised deep learning.
• Employed a multi-level Mixture-of-Experts approach combining distortion-aware perceptual and content-aware deep features.
• Utilized a self-supervised, pre-trained deep learning network for high-level semantic feature extraction of human avatars.

Main Results:

• HoloQA achieved state-of-the-art performance on the LIVE-Meta Rendered Human Avatar VQA database.
• Demonstrated efficacy in predicting the quality of rendered human avatars in VR.
• Showcased competitive performance on other digital human avatar datasets and cloud gaming video quality assessment.

Conclusions:

• HoloQA represents a significant advancement in VQA for digital human avatars in immersive technologies.
• The model's design effectively captures unique distortions, outperforming standard VQA approaches.
• HoloQA shows promise for applications beyond VR/AR, including cloud gaming, and its code will be publicly available.