RLStereo: Real-Time Stereo Matching Based on Reinforcement Learning

Many state-of-the-art stereo matching algorithms based on deep learning have been proposed in recent years, which usually construct a cost volume and adopt cost filtering by a series of 3D convolutions. In essence, the possibility of all the disparities is exhaustively represented in the cost volume, and the estimated disparity holds the maximal possibility. The cost filtering could learn contextual information and reduce mismatches in ill-posed regions. However, this kind of methods has two main disadvantages: 1) cost filtering is very time-consuming, and it is thus difficult to simultaneously satisfy the requirements for both speed and accuracy; 2) thickness of the cost volume determines the disparity range which can be estimated, and the pre-defined disparity range may not meet the demand of practical application. This paper proposes a novel real-time stereo matching method called RLStereo, which is based on reinforcement learning and abandons the cost volume or the routine of exhaustive search. The trained RLStereo makes only a few actions iteratively to search the value of the disparity for each pair of stereo images. Experimental results show the effectiveness of the proposed method, which achieves comparable performances to state-of-the-art algorithms with real-time speed on the public large-scale testset, i.e., Scene Flow.