Human-in-the-loop Extraction of Interpretable Concepts in Deep Learning Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) are increasingly used for critical decisions, necessitating effective interpretation methods.
• Concept-based explanations are a popular post-hoc interpretation technique for DNNs.
• Automatically identifying human-understandable visual concepts influencing DNN decisions remains challenging.

Purpose of the Study:

• To present a novel human-in-the-loop approach for generating user-defined concepts for DNN interpretation and diagnostics.
• To integrate active learning for efficient concept extraction with minimal human labeling.
• To develop an interactive system, ConceptExtract, for analyzing model behavior and refining models.

Main Methods:

• Utilizing active learning to combine human knowledge and feedback for training a concept extractor.
• Developing an interactive system (ConceptExtract) to facilitate concept generation and model analysis.
• Applying the approach to diverse machine learning tasks and datasets through case studies.

Main Results:

• Demonstrated the ability to extract meaningful and human-friendly visual concepts.
• Showcased how extracted concepts aid in understanding predictions and comparing model performance.
• Identified visual concepts negatively impacting model performance, leading to targeted data augmentation.
• Quantitative experiments confirmed the accuracy of concept extraction via active learning.

Conclusions:

• The proposed human-in-the-loop active learning approach effectively extracts human-understandable concepts for DNN interpretation.
• ConceptExtract enables better analysis of model behavior, prediction understanding, and model refinement.
• Identifying detrimental concepts allows for data augmentation strategies that consistently improve model performance.