R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning Explainability

Background:

• Transformer models are increasingly used in natural language processing (NLP), computer vision, and multi-modal applications like GPT4.
• Enhancing the explainability of these powerful models is crucial for building trust and enabling deeper understanding in downstream tasks.
• Current methods often lack the granularity needed to pinpoint specific feature contributions to image classification.

Purpose of the Study:

• To develop a novel method for improving the explainability of transformer-based image classification models.
• To provide users with visualizations of class-specific maps for better comprehension of model predictions.
• To increase trust in AI-driven image classification results.

Main Methods:

• Introduced two novel modules: 'Relationship Weighted Out' for extracting class-specific intermediate features and 'Cut' for fine-grained feature decomposition (position, texture, color).
• Integrated these modules to generate dense, class-specific visual explainability maps.
• Validated the approach using extensive qualitative and quantitative experiments on ImageNet and the LRN dataset (automatic driving danger alerts).

Main Results:

• The proposed method significantly improved explainability compared to previous approaches on both standard and complex datasets.
• Ablation experiments confirmed the effectiveness and individual contributions of both the 'Relationship Weighted Out' and 'Cut' modules.
• Generated detailed visual maps that clearly highlight relevant features for specific classes.

Conclusions:

• The novel method effectively enhances the explainability of transformer-based image classifiers.
• The generated class-specific maps provide valuable insights into model decision-making processes.
• This approach offers a robust solution for increasing trust and understanding in AI-powered image analysis, particularly in safety-critical applications.