Similarity and quality metrics for MR image-to-image translation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Quantitative Analysis

Background:

• Image-to-image translation in medical imaging generates synthetic images for various applications.
• Human validation of synthetic images is time-consuming and costly.
• Quantitative metrics offer objective and reproducible assessment.

Purpose of the Study:

• To quantitatively analyze the sensitivity of reference and non-reference image quality metrics.
• To evaluate metrics for assessing synthetic medical images generated by image-to-image translation.
• To provide recommendations for effective metric usage in AI model evaluation.

Main Methods:

• Analyzed 11 similarity (reference) and 12 quality (non-reference) metrics.
• Investigated sensitivity to 11 distortion types and MR artifacts.
• Assessed the influence of normalization methods on metric performance.
• Included a metric evaluating a downstream segmentation task.

Main Results:

• Identified varying sensitivities of metrics to specific distortions and artifacts.
• Demonstrated the impact of normalization techniques on metric reliability.
• Highlighted limitations of traditional metrics like SSIM and PSNR for certain distortions.
• Showcased the utility of non-reference metrics for distortion detection.

Conclusions:

• No single metric is universally optimal for evaluating all synthetic medical images.
• Careful selection of metrics based on expected distortions and available references is crucial.
• Recommendations are provided for selecting appropriate similarity and quality metrics for image-to-image translation models.