EGOF-Net: epipolar guided optical flow network for unrectified stereo matching

It is challenging to realize stereo matching in dynamic stereo vision systems. We present an epipolar guided optical flow network (EGOF-Net) for unrectified stereo matching by estimating robust epipolar geometry with a deep cross-checking-based fundamental matrix estimation method (DCCM) and then surpassing false matches with a 4D epipolar modulator (4D-EM) module. On synthetic and real-scene datasets, our network outperforms the state-of-the-art methods by a substantial margin. Also, we test the network in an existing dynamic stereo system and successfully reconstruct the 3D point clouds. The technique can simplify the stereo vision pipeline by ticking out rectification operations. Moreover, it suggests a new opportunity for combining heuristic algorithms with neural networks. The code is available on https://github.com/psyrocloud/EGOF-Net.