RPCANet$^{++}$: Deep Interpretable Robust PCA for Sparse Object Segmentation

Robust principal component analysis (RPCA) decomposes an observation matrix into low-rank background and sparse object components. This capability has enabled its application in tasks ranging from image restoration to segmentation. However, traditional RPCA models suffer from computational burdens caused by matrix operations, reliance on finely tuned hyperparameters, and rigid priors that limit adaptability in dynamic scenarios. To overcome these limitations, we propose RPCANet⁺⁺, a sparse object segmentation framework that fuses the interpretability of RPCA with efficient deep architectures. By explicitly mapping the iterative optimization steps to network layers, we bridge the gap between physical interpretability and deep representation learning. Our approach unfolds a relaxed RPCA model into a structured network comprising a Background Approximation Module (BAM), an Object Extraction Module (OEM), and an Image Restoration Module (IRM). To mitigate inter-stage transmission loss in BAM, we introduce a Memory-Augmented Module (MAM) to enhance background feature preservation, while a Deep Contrast Prior Module (DCPM) leverages saliency cues to expedite object extraction. Extensive experiments on diverse datasets demonstrate that RPCANet⁺⁺ achieves state-of-the-art performance on Infrared Small Target Detection (IRSTD) and Vessel Segmentation (VS) tasks, while maintaining competitive results on Defect Detection (DD) tasks involving larger targets. We further improve interpretability via visual and numerical low-rankness and sparsity measurements. By combining the theoretical strengths of RPCA with the efficiency of deep networks, our approach sets a new baseline for reliable and interpretable sparse object segmentation. Codes are available at our Project Webpage.