SAMScore: A Content Structural Similarity Metric for Image Translation Evaluation

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Image translation, including style transfer and modality conversion, aims for realistic and faithful image generation.
• Preserving content structure in image translation remains a significant challenge.
• Existing image-level similarity metrics often fail to capture high-level structural content accurately.

Purpose of the Study:

• Introduce SAMScore, a novel content structural similarity metric for evaluating image translation models.
• Address the limitations of traditional metrics in assessing structural faithfulness.
• Provide a more accurate evaluation tool for the field of image translation.

Main Methods:

• Leveraged the high-performance Segment Anything Model (SAM) to develop SAMScore.
• Designed SAMScore as a generic metric applicable across various image translation tasks.
• Compared SAMScore against existing metrics on 19 diverse image translation tasks.

Main Results:

• SAMScore demonstrated superior performance in evaluating faithfulness across all tested image translation tasks.
• The metric achieved standout accuracy in content similarity comparisons.
• SAMScore outperformed all competitive metrics in the evaluation.

Conclusions:

• SAMScore is a valuable new tool for the precise evaluation of image translation models.
• The metric facilitates more accurate assessments of realism and structural faithfulness.
• SAMScore is expected to drive advancements in the field of image translation.