A Deep Multi-modal Explanation Model for Zero-shot Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot learning (ZSL) enables classification of unseen classes by learning visual and semantic embeddings.
• Generating explanations for ZSL decisions remains an underexplored area.
• Explainable AI (XAI) is crucial for understanding and trusting machine learning models.

Purpose of the Study:

• To introduce and address the novel task of explainable zero-shot learning (XZSL).
• To develop a model that generates both accurate ZSL classifications and supporting visual/textual explanations.
• To investigate the interplay between classification performance and explanation quality in ZSL.

Main Methods:

• Proposed a Deep Multi-modal Explanation (DME) model with joint visual-attribute embedding and multi-channel explanation modules.
• Developed attribute activation maps (AAM) and class activation maps (CAM) for visual explanations.
• Utilized three Long Short-Term Memory (LSTM) models for generating diverse textual explanations.

Main Results:

• The DME model achieved competitive ZSL classification performance on benchmark datasets.
• Ablation studies confirmed the effectiveness of individual model components.
• Demonstrated that joint training improves both classification and explanation capabilities.

Conclusions:

• The DME model successfully integrates classification and explanation for ZSL.
• The approach offers insights into model interpretability and decision-making processes.
• XZSL holds promise for enhancing trust and transparency in AI systems.