Image2Brain: a cross-modality model for blind stereoscopic image quality ranking

Area of Science:

• Computer Vision
• Neuroscience
• Signal Processing

Background:

• Human perception of stereoscopic image quality is complex, involving the cerebral visual cortex.
• Traditional stereoscopic image quality assessment methods primarily focus on image features, neglecting human perceptual mechanisms.
• Developing machine-based methods to replicate human perception from electroencephalogram (EEG) signals offers a more accurate approach.

Purpose of the Study:

• To propose a novel image-to-brain (I2B) cross-modality model for accurate stereoscopic image quality assessment.
• To develop a system that emulates human visual perception mechanisms using EEG signals.
• To enable the prediction of image perceptual quality by converting image features into brain representations.

Main Methods:

• A spatial-temporal EEG encoder (STEE) was developed to learn EEG representations.
• An I2B deep convolutional generative adversarial network (I2B-DCGAN) was utilized, incorporating a semantic-guided image encoder.
• The model generates corresponding EEG features for stereoscopic images, which are then classified to predict perceptual quality.

Main Results:

• The proposed I2B cross-modality model demonstrated superior performance in emulating human brain visual perception.
• The method achieved an average accuracy of 95.95% on a brain-visual multimodal stereoscopic image quality ranking database.
• Experimental results indicate the model outperforms existing stereoscopic image quality assessment methods.

Conclusions:

• The developed method effectively converts learned stereoscopic image features into brain representations without requiring EEG signals during testing.
• The proposed model exhibits good generalization ability across new datasets.
• The research highlights the potential for practical applications in stereoscopic image quality assessment.