Beyond distortions: a benchmark for subjective evaluation of image rendering quality

Area of Science:

• Computer Vision
• Image Processing
• Human-Computer Interaction

Background:

• Traditional Image Quality Assessment (IQA) focuses on technical degradations like noise and blur.
• The aesthetic quality of rendered images is primarily determined by color processing, not technical flaws.
• Quantitative evaluation of rendering methods' impact on image quality remains underdeveloped.

Purpose of the Study:

• Introduce Image Rendering Quality Assessment (IRQA) as a new problem within IQA.
• Present REPID, a large-scale benchmark for studying IRQA.
• Investigate content-dependent render preferences and rendering parameter influences.

Main Methods:

• Collected 30,000 edited images with preference annotations from 13,648 voters (over 2.5 million votes).
• Developed IRQA-specific models and compared them against traditional IQA metrics, handcrafted features, deep learning, and foundation-model embeddings.
• Explored applications including aesthetic preference prediction, render ranking, and benchmarking aesthetic evaluation methods.

Main Results:

• IRQA-specific models achieved up to 40% higher precision on the REPID benchmark compared to conventional IQA methods.
• Investigated content-dependent rendering preferences and the impact of rendering parameters.
• Demonstrated the effectiveness of IRQA models in aesthetic preference prediction and render ranking.

Conclusions:

• Image Rendering Quality Assessment (IRQA) is a crucial new direction for evaluating aesthetic image quality.
• The REPID benchmark provides a valuable resource for advancing IRQA research.
• IRQA-specific models offer superior performance for assessing rendered image aesthetics over traditional IQA approaches.