Local-Global Aware Concept Bottleneck Models for Interpretable Image Classification.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Concept Bottleneck Models (CBMs) offer interpretable image classification but struggle with CLIP's biases.
• CLIP's global representation and lack of region sensitivity limit CBMs in critical sensor-driven fields.
• Remote sensing and medical imaging require localized visual evidence for accurate classification.

Purpose of the Study:

• To develop an improved Concept Bottleneck Model (CBM) addressing CLIP's limitations.
• To enhance concept prediction accuracy and reliability for interpretable image classification.
• To create a model suitable for sensor-driven applications demanding localized visual understanding.

Main Methods:

• Proposed the Local-Global Aware Concept Bottleneck Model (LGA-CBM) with a training-free refinement pipeline.
• Introduced Dual Masking Guided Concept Score Refinement (DMCSR) using attention weights for region-concept alignment.
• Implemented Local-to-Global Concept Reidentification (L2GCR) and Similar Concepts Correction Mechanism (SCCM) with Grounding DINO for disambiguation.

Main Results:

• LGA-CBM demonstrated state-of-the-art performance across six benchmark datasets.
• Achieved superior accuracy and interpretability compared to existing methods.
• Generated explanations that closely align with human cognitive understanding.

Conclusions:

• LGA-CBM effectively refines concept scores, overcoming CLIP's limitations.
• The model provides highly interpretable image classification with minimal concept usage.
• LGA-CBM shows significant promise for applications in remote sensing and medical imaging.