Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• The increasing use of image-based applications necessitates accurate and interpretable image similarity measures.
• Current models often lack transparency, hindering understanding of similarity judgments.

Purpose of the Study:

• To develop an explainable image similarity approach providing both scores and visual explanations.
• To enhance the interpretability and trustworthiness of image-based systems.

Main Methods:

• Integration of Siamese Networks for feature extraction.
• Application of Gradient-weighted Class Activation Mapping (Grad-CAM) for visual explanations.
• Development of a framework for generating factual and counterfactual explanations.

Main Results:

• The proposed framework successfully generates similarity scores with accompanying visual explanations.
• Demonstrated potential for providing factual and counterfactual insights into image similarity.
• The approach facilitates better decision-making by clarifying similarity reasoning.

Conclusions:

• Explainable image similarity enhances interpretability, trustworthiness, and user acceptance.
• The framework offers a novel method for understanding image comparisons in AI.
• Addresses the critical need for transparency in image-based AI applications.