Outlier-Aware Contrastive Learning

Contrastive learning aims to learn an embedding space with sample discrimination where similar samples attract together while dissimilar samples repulse apart. However, the issue of sampling bias likely happens and degrades the classification performance when a contrast model is trained with the leakage caused by similar samples but from different classes or dissimilar samples from the same class. Out-of-distribution (OOD) detection provides a meaningful scheme to detect and mask those false negative samples for debiasing in an outlier-aware contrastive loss for high-fidelity contrastive learning. Sample debiasing is feasible to reduce the upper bound of contrastive loss. Also, the previous OOD detector was trained from auxiliary collection of OOD samples. In real world, the prior knowledge of OOD samples is commonly unavailable. This study presents new outlier-aware detection and contrast models through generation and augmentation of those samples near the boundary between in-distribution (ID) and OOD. These synthesized samples are located right outside ID, and their Gaussian embeddings sufficiently reflect OOD behaviors. An OOD detector is learned by using ID samples and synthesized OOD samples with the learning objective towards contrastive OOD detection and debiased contrast model. The experiments are conducted to illustrate the merit of the proposed outlier-aware contrastive learning.