Study of Subjective and Objective Quality Assessment of Audio-Visual Signals

Area of Science:

• Multimedia systems
• Signal processing
• Human-computer interaction

Background:

• Traditional quality assessment (QA) focuses on single modalities (audio or video) despite their frequent co-occurrence.
• Distortions in audio and video signals jointly impact user-perceived quality of experience (QoE).

Purpose of the Study:

• To develop and validate objective models for predicting audio-visual (A/V) quality.
• To create a comprehensive subjective A/V quality assessment database for research.

Main Methods:

• Generated 336 A/V sequences with 24 distortion combinations from 14 sources.
• Conducted a subjective A/V quality perception study.
• Designed four families of objective A/V quality prediction models using multimodal fusion strategies, incorporating deep neural networks and various fusion techniques.

Main Results:

• Validated objective A/V quality prediction models against the new subjective database.
• Demonstrated the effectiveness of multimodal fusion strategies for A/V quality assessment.

Conclusions:

• The developed objective models provide accurate predictions of A/V quality.
• The publicly released LIVE-SJTU A/V-QA Database will advance research in multimodal quality assessment.