Diffusion Semantic Segmentation: A generative segmentation model based on joint distributions

The mainstream semantic segmentation schemes are essentially discriminative paradigms based on conditional distributions $p(class\mid feature)$ . Although efficient and straightforward, this prevalent paradigm ignores the latent data distribution $p(feature\mid calss)$ and it is difficult to identify out-of-distribution data based on pixel feature space in a simple manner. Going beyond the concept of out-of-distribution, we propose a novel segmentation architecture based on joint distribution, called the Denoising Semantic Segmentation Model (DSSM). We propose learning classification decision boundaries in pixel feature space and modeling joint distributions in label feature space. Specifically, DSSM optimizes probability maps based on pixel feature classification through Bayesian posterior probability. To that end, we design a Fusion Information Module (FIM) to guide the generative module in inference and provide latent label features for the semantic module. Furthermore, we introduce a stable Markov inference process to reduce inference offset. Finally, the joint distribution based model is end-to-end trained in a discriminative manner, i.e., maximizing $p(class\mid feature)$ , which endows DSSM with the strengths of both generative and discriminative models. DSSM is evaluated on five medical image datasets and is shown to perform better than state-of-the-art discriminative models. What is even more impressive is that DSSM still performs well on small datasets, so we believe that this work will bring a new insight into related fields.