A Perspective on Quality Evaluation for AI-Generated Videos

Area of Science:

• Artificial Intelligence
• Computer Vision
• Multimedia Processing

Background:

• AI-generated content (AIGC) has advanced video creation, but reliable quality evaluation is challenging.
• Current methods struggle with spatial fidelity, temporal coherence, and semantic alignment in machine-crafted videos.
• Sensor technologies are crucial for ensuring the physical plausibility of AIGC outputs.

Purpose of the Study:

• To propose multimodal large language models (MLLMs) as a cornerstone for next-generation video quality assessment (VQA).
• To highlight the potential of MLLMs in overcoming limitations of traditional VQA methods.
• To analyze current AIGC video quality assessment methodologies.

Main Methods:

• Utilizing MLLMs to jointly encode multi-modal cues (vision, language, sound, depth).
• Leveraging MLLM's language understanding for assessing scene composition, motion dynamics, and narrative consistency.
• Analyzing existing AIGC generation models, datasets, quality dimensions, and evaluation frameworks.

Main Results:

• MLLMs can overcome the fragmentation of hand-engineered metrics and poor generalization of CNN-based methods.
• Advances in sensor fusion enable MLLMs to integrate physical constraints with semantic interpretations.
• Enhanced accuracy in visual quality assessment through combined low-level and high-level feature analysis.

Conclusions:

• MLLMs represent a significant advancement for robust and comprehensive AIGC video quality assessment.
• The integration of multi-modal data and sensor fusion is key to future VQA systems.
• Future research should focus on developing and refining MLLM-based VQA frameworks.