Dual-Masked and Discriminative Reconstruction for Unified Vision Anomaly Detection

Unsupervised reconstruction networks have shown promise for unified vision anomaly detection, i.e., image-level anomaly classification and pixel-level anomaly segmentation, where a single model trained on multi-class normal images can detect various anomalies. This is more challenging than most existing separate methods, i.e., one model for one class, as it requires handling a more complex data distribution. Notably, pure reconstruction networks often suffer from overfitting due to "identity shortcut", where both normal and anomaly images may be well recovered and thus fail in detecting anomalies. Recent efforts have focused on developing specific modules for different network architectures, e.g., Convolutions and Transformers. However, it is still unclear how to essentially and effectively prevent learning from this shortcut in a simpler and more general manner. Furthermore, most existing methods consider anomaly detection solely as unsupervised classification, resulting in inaccurate anomaly segmentation due to "weak discrimination", where normal and anomaly features may be entangled. To address these challenges, we propose a simple yet general Dual-masked and Discriminative Reconstruction (D2Rec) for unified vision anomaly detection. First, we propose a general dual-masked reconstruction, i.e., using a pair of complementary masks, resolving the "identity shortcut" so that all masked positions are reconstructed by unmasked original features. Second, we propose a self-supervised discriminator, which refines reconstruction errors with synthesized anomaly images to enhance the discrimination ability between normal and abnormal features. The dual-masked reconstruction and self-supervised discriminator can serve as universal plugins, easily integrated into reconstruction-based anomaly detection methods of any architecture. Despite its simplicity, D2Rec outperforms previous methods on three industrial benchmarks (MVTec, BTAD, and VisA), and three medical datasets (Brain MRI, Liver CT and Retinal OCT). The code for D2Rec is available at https://github.com/gaobb/D2Rec.