Evaluating Quantitative Metrics of Tone-Mapped Images

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Subjective evaluation of tone-mapped images is time-consuming.
• Existing automatic image quality assessment metrics are often evaluated on limited datasets, risking overfitting.
• There is a need for robust evaluation of image quality metrics across diverse scenes.

Purpose of the Study:

• To propose a novel framework for evaluating image quality assessment metrics without subjective data.
• To assess the performance of existing metrics and identify their limitations.
• To develop a method for ranking the relative performance of different metrics.

Main Methods:

• Utilizing population-based metaheuristics to evaluate metric performance.
• Synthesizing realistic tone-mapped images by modifying tone-mapping curves, not image pixels.
• Treating evaluated metrics as black boxes, allowing seamless replacement of any metric.
• Developing a competitive framework where metrics identify errors in each other's scores.

Main Results:

• Evaluated six existing image quality assessment metrics.
• Synthesized images where current metrics failed to assign accurate visual quality scores.
• Demonstrated the framework's ability to identify metric weaknesses across various scenes.
• Proposed a method to rank the relative performance of evaluated metrics.

Conclusions:

• The proposed framework offers an effective, automated approach to evaluate image quality assessment metrics.
• This method overcomes the limitations of dataset-specific evaluations and overfitting.
• The flexible, black-box design allows for easy integration and evaluation of future metrics.