From Generation to Optimization: Improving Pseudo Labels for Semi-Supervised Object Detection

In semi-supervised object detection (SSOD), due to the limited availability of labeled data, the quality and quantity of pseudo labels generated from unlabeled images are crucial for model training. Our study reveals that in the early stages of training, the number of usable pseudo labels is very low, which hampers the model's ability to converge quickly. Furthermore, throughout the training process, even though pseudo-labels are generated with high confidence, a significant number of labeling errors still occur. To address these challenges, we propose a novel framework, Generation-to-Optimization of Pseudo Labels (GO-PL), which comprehensively improves the model's performance in both the pseudo labels generation stage and the pseudo labels optimization stage. Specifically, we design a CLIP-based Pseudo Label Generation (CPLG) module, which matches proposals from the teacher model with CLIP text embeddings to generate semantically consistent high-confidence pseudo labels. These labels are further integrated into student training via an auxiliary loss, enabling early-stage guidance with stable and informative supervision signals. We also introduce Pixel-guided Instance Weighting (PIW) module that utilizes the SAM to assess the pixel-level quality of pseudo labels generated by the teacher model. Combined with a local voting mechanism, PIW dynamically assigns weights to each pseudo-label, thereby optimizing the impact of labels with varying quality on gradient updates. GO-PL demonstrates strong performance on standard SSOD benchmarks, including the MS-COCO and PASCAL VOC datasets. Notably, it achieves 26.7% mAP with only 1% labeled data on the MS-COCO, surpassing the previous best approach by 1.7%.