Digging into Monocular Texture-Scale Feature Fusion for Optical Flow and Optical Expansion Estimation

Visual perception in dynamic scenes is a crucial task in the fields of computer vision and autonomous driving. In this paper, we focus on monocular optical flow and expansion estimation. Optical flow describes the positional changes of pixels between two frames, while optical expansion describes local image-scale changes around pixels and their neighbourhoods. Monocular optical flow estimation has a well-established research foundation, with most existing methods achieving high-precision optical flow through image texture feature extraction and matching. However, research on optical expansion is limited. Existing optical expansion estimation methods all rely on optical flow results or model structures and do not accurately model the expansion task itself. In fact, object motion in dynamic scenes generates rich local scale information in images, which is an ideal characteristic for solving the optical expansion task. In this paper, we propose and explore methods for extracting and interacting image texture-scale features and apply them to achieve high-quality monocular optical flow-expansion estimation. We propose MonoFusion, a new formulation and learning framework for jointly estimating optical flow and optical expansion by independently extracting and fusing texture and scale features. Additionally, we introduce MonoFusion-R, which enriches the texture-scale fusion process with a recurrent framework. Our method achieves state-of-the-art (SOTA) monocular performance on four KITTI tasks, including optical flow, optical expansion, time-to-collision, and scene flow, and also shows strong optical-flow generalization on FlyingThings3D.