Interpretable Failure Detection with Human-Level Concepts

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Reliable failure detection is crucial for safety-critical applications.
• Neural networks often exhibit overconfident predictions for misclassified data.
• Current failure detection methods using category-level signals (logits) are insufficient.

Purpose of the Study:

• To develop a novel strategy for reliable neural network failure detection.
• To enable transparent interpretation of the reasons behind model failures.
• To improve the accuracy of confidence scores in image classification.

Main Methods:

• Leveraging human-level concepts for failure analysis.
• Integrating nuanced signals for each category for finer-grained confidence assessment.
• Utilizing ordinal ranking of concept activation to input images.

Main Results:

• Significantly reduced false positive rates across diverse benchmarks.
• Achieved a 3.7% reduction in false positives on ImageNet.
• Achieved a 9% reduction in false positives on EuroSAT.

Conclusions:

• The proposed concept-based approach offers a simple yet highly effective solution for failure detection.
• This method enhances model transparency by explaining failure causes.
• The approach demonstrates superior performance in reducing false positives in real-world image classification.