Stereo matching via self-supervised pre-training with view synthesis-based pretext task

Recently, deep learning-based stereo matching has made significant progress. However, most previous methods heavily rely on large-scale labeled images for training, which limits the application of stereo matching. In this paper, a Self-supervised Pre-training method with a View synthesis-based Pretext task (SPVP) is proposed for stereo matching to alleviate the problem of labeled data dependency. Specifically, to explore the corresponding relationships between image pairs, a view synthesis-based pretext task is constructed to learn the mapping between the left and right views. In the proposed pretext task, a monocular-part network is developed to complete the conversion from right views to pseudo-left views, and a stereo-part network is constructed to explore the corresponding relationships between image pairs. In the main task (i.e., stereo matching), through utilizing the pre-trained model from the pretext task, the corresponding relationship learned by the view synthesis-based pretext task is transferred to the network, thus enabling the network to be trained using a small amount of labeled data. Extensive experimental results demonstrate the effectiveness of the proposed SPVP.