Describe, Spot and Explain: Interpretable Representation Learning for Discriminative Visual Reasoning.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Deep neural networks (DNNs) achieve high performance but lack transparency in decision-making.
• Explainable AI (XAI) is crucial for understanding complex deep learning models.
• Current XAI methods struggle to provide intuitive explanations for DNNs.

Purpose of the Study:

• To propose a novel representation learning framework, Describe, Spot and eXplain (DSX), for enhancing model interpretability.
• To enable semantic interpretability of classification tasks using deep learning models.
• To offer practical visual explanations associated with physical attributes.

Main Methods:

• Developed a Transformer-based framework (DSX) with two stages: descriptive prototype learning and discriminative prototype discovery.
• The descriptive stage derives visual representations from input images.
• The discriminative stage identifies a subset of representations for interpretability, without requiring ground truth attribute supervision.

Main Results:

• DSX successfully generates semantically meaningful visual representations.
• The framework provides practical interpretability by associating representations with domain-expert-provided attributes.
• Experiments on fine-grained classification and person re-identification demonstrate satisfactory recognition performance and interpretability.

Conclusions:

• The DSX framework offers a novel approach to explainable AI for deep learning models.
• DSX achieves a balance between recognition performance and semantic interpretability.
• The method's ability to associate learned representations with physical attributes enhances practical applicability.