ESIQA: Perceptual Quality Assessment of Vision-Pro-based Egocentric Spatial Images.

Area of Science:

• Computer Vision
• Virtual Reality
• Image Processing

Background:

• eXtended Reality (XR) and head-mounted displays (HMDs) are advancing, with egocentric spatial images becoming a key content type.
• Assessing the Quality of Experience (QoE) for XR content is crucial, but egocentric spatial images present unique challenges for traditional Image Quality Assessment (IQA).
• Existing IQA research has not adequately addressed the specific characteristics of egocentric spatial images.

Purpose of the Study:

• To establish the first dedicated Image Quality Assessment (IQA) database for egocentric spatial images.
• To propose a novel deep learning model for predicting the perceptual quality of egocentric spatial images.
• To evaluate the model's performance across different display modes relevant to XR.

Main Methods:

• Creation of the Egocentric Spatial Images Quality Assessment Database (ESIQAD) with 500 images and Mean Opinion Scores (MOSs) for 2D, 3D-window, and 3D-immersive displays.
• Development of ESIQAnet, a mamba2-based multi-stage feature fusion model utilizing Visual State Space Duality (VSSD) blocks, cross-attention, and transposed attention.
• Feature extraction, fusion of binocular view information, feature refinement, and quality regression for perceptual quality prediction.

Main Results:

• The ESIQAnet model demonstrated superior performance compared to 22 state-of-the-art IQA models.
• The model achieved high accuracy in predicting perceptual quality across all three tested display modes (2D, 3D-window, 3D-immersive).
• The established ESIQAD provides a valuable resource for future research in egocentric spatial image quality.

Conclusions:

• The proposed ESIQAnet model effectively predicts the perceptual quality of egocentric spatial images in XR.
• The ESIQAD is a significant contribution to the field, enabling further development and benchmarking of IQA methods for immersive content.
• This work addresses a critical gap in IQA research for emerging XR visual media.