Visual Quality Evaluation for Semantic Segmentation: Subjective Assessment Database and Objective Assessment Measure.

Area of Science:

• Computer Vision
• Image Analysis
• Machine Learning

Background:

• Quality evaluation is crucial for advancing semantic segmentation applications.
• Existing objective metrics may not fully align with human perception of segmentation quality.

Purpose of the Study:

• To establish a subjective quality assessment database for semantic segmentation.
• To develop a novel objective metric that better reflects subjective quality judgments.
• To validate the reliability of the proposed database and metric.

Main Methods:

• A subjective semantic segmentation quality assessment database was created using the stimulus-comparison method.
• A robust regression mapping model was employed to link subjective assessments with objective distances.
• A novel Relative Quality Prediction Network (RQPN) based on Siamese Convolutional Neural Networks (CNN) was proposed.

Main Results:

• The developed database provides relative quality assessments for semantic segmentation result pairs.
• The regression model helps in examining the alignment between objective metrics and subjective judgments.
• The RQPN, trained on the subjective database, showed higher consistency with subjective assessments than existing objective metrics.

Conclusions:

• The proposed subjective database and RQPN offer a more reliable method for evaluating semantic segmentation algorithms.
• RQPN can assess algorithms not included in the training database, enhancing its generalizability.
• This work contributes to the development of more perceptually aligned objective metrics for semantic segmentation.