Double nuclear norm-based matrix decomposition for occluded image recovery and background modeling

Robust principal component analysis (RPCA) is a new emerging method for exact recovery of corrupted low-rank matrices. It assumes that the real data matrix has low rank and the error matrix is sparse. This paper presents a method called double nuclear norm-based matrix decomposition (DNMD) for dealing with the image data corrupted by continuous occlusion. The method uses a unified low-rank assumption to characterize the real image data and continuous occlusion. Specifically, we assume all image vectors form a low-rank matrix, and each occlusion-induced error image is a low-rank matrix as well. Compared with RPCA, the low-rank assumption of DNMD is more intuitive for describing occlusion. Moreover, DNMD is solved by alternating direction method of multipliers. Our algorithm involves only one operator: the singular value shrinkage operator. DNMD, as a transductive method, is further extended into inductive DNMD (IDNMD). Both DNMD and IDNMD use nuclear norm for measuring the continuous occlusion-induced error, while many previous methods use L1 , L2 , or other M-estimators. Extensive experiments on removing occlusion from face images and background modeling from surveillance videos demonstrate the effectiveness of the proposed methods.