Learning by Seeing More Classes

Traditional pattern recognition models usually assume a fixed and identical number of classes during both training and inference stages. In this paper, we study an interesting but ignored question: can increasing the number of classes during training improve the generalization and reliability performance? For a k-class problem, instead of training with only these k classes, we propose to learn with k+m classes, where the additional m classes can be either real classes from other datasets or synthesized from known classes. Specifically, we propose two strategies for constructing new classes from known classes. By making the model see more classes during training, we can obtain several advantages. First, the added m classes serve as a regularization which is helpful to improve the generalization accuracy on the original k classes. Second, this will alleviate the overconfident phenomenon and produce more reliable confidence estimation for different tasks like misclassification detection, confidence calibration, and out-of-distribution detection. Lastly, the additional classes can also improve the learned feature representation, which is beneficial for new classes generalization in few-shot learning and class-incremental learning. Compared with the widely proved concept of data augmentation (dataAug), our method is driven from another dimension of augmentation based on additional classes (classAug). Comprehensive experiments demonstrated the superiority of our classAug under various open-environment metrics on benchmark datasets.