The Transform-and-Perform Framework: Explainable Deep Learning Beyond Classification

Area of Science:

• Computer Science
• Artificial Intelligence
• Data Visualization

Background:

• Deep learning (DL) models are increasingly used for complex high-dimensional-to-high-dimensional (H-H) problems like image translation.
• Existing visual analytics (VA) systems primarily focus on classification tasks, limiting their applicability to H-H problems.
• H-H problems lack distinct classes, requiring analysis of continuous, non-linear input-output relationships.

Purpose of the Study:

• To address the limitations of current VA systems in analyzing DL models for H-H problems.
• To introduce a unified conceptual framework, Transform-and-Perform (T&P), for designing effective VA systems for H-H DL models.
• To provide a structured approach for understanding existing VA systems and identifying areas for improvement.

Main Methods:

• Development of the Transform-and-Perform (T&P) framework.
• Utilizing T&P as a checklist to structure analysis workflows and strategies.
• Applying the T&P framework to a real-world image-to-image translation application.

Main Results:

• The T&P framework facilitates the design of VA systems tailored for H-H DL problems.
• T&P aids in a structured understanding of DL models by analyzing input, model, and output relationships.
• The framework effectively supports the identification of gaps in current VA systems for H-H tasks.

Conclusions:

• A novel framework (T&P) is necessary for advancing VA in complex DL applications beyond classification.
• The T&P framework enhances the development of effective VA tools for model interpretability in H-H problems.
• This work highlights the need for specialized frameworks to unlock the full potential of DL in diverse applications.