InDeed: Interpretable Image Deep Decomposition With Grounded Generalizability

Image decomposition aims to analyze an image into elementary components, which is essential for numerous downstream tasks and also by nature provides certain interpretability to the analysis. Deep learning can be powerful for such tasks, but surprisingly their combination with a focus on interpretability and generalizability is rarely explored. In this work, we introduce a novel framework to decompose an image into the low-rank, sparse, and noise components, combining hierarchical Bayesian modeling and deep learning to create an architecture-modularized and model-generalizable neural network (DNN). The proposed framework includes three steps: (1) hierarchical Bayesian modeling of image decomposition, (2) transforming the inference problem into optimization tasks, and (3) deep inference via a modularized Bayesian DNN under a relaxed amortized formulation. We further analyze the connection between the loss function and the generalization error bound following the PAC-Bayes theory, which then motivates a new test-time adaptation approach for out-of-distribution scenarios. We instantiated the application using two downstream tasks, i.e., image denoising and unsupervised anomaly detection, and the results demonstrated improved generalizability as well as interpretability of our methods. The source code is available at https://github.com/LucyyyyW/InDeed.