Inverse-Based Approach to Explaining and Visualizing Convolutional Neural Networks.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing methods for analyzing Convolutional Neural Networks (CNNs) primarily focus on global scores and pixelwise input contributions.
• Analysis of CNNs for multi-labeled outputs or regression tasks remains underexplored, despite their success in image classification.

Purpose of the Study:

• To propose a novel inverse-based approach for understanding and visualizing CNNs.
• To extend CNN analysis to regression and multi-label classification tasks.
• To improve the interpretability and diagnostic capabilities of CNN models.

Main Methods:

• Developed a layerwise inverse procedure by computing the inverse of a feedforward pass.
• Incorporated two key observations: consistency of internal activations and desirability of minimal activation for interpretability.
• Applied the method to both image classification and regression tasks, including single image super-resolution.

Main Results:

• The proposed method enables unified analysis of CNNs for classification and regression.
• Attribution analysis for image classification achieved performance comparable to state-of-the-art methods.
• A novel visualization plot was developed to illustrate the trade-off between activation levels and class re-identification rates.
• Analysis of super-resolution CNNs revealed overlooked frequency bands potentially degrading performance.

Conclusions:

• The inverse-based approach provides a versatile framework for CNN interpretability across different tasks.
• The method offers insights into model behavior, aiding in performance diagnostics and improvement.
• Future work can leverage this framework for more sophisticated CNN analysis and development.