Errors-in-variables modeling in optical flow estimation

Gradient-based optical flow estimation methods typically do not take into account errors in the spatial derivative estimates. The presence of these errors causes an errors-in-variables (EIV) problem. Moreover, the use of finite difference methods to calculate these derivatives ensures that the errors are strongly correlated between pixels. Total least squares (TLS) has often been used to address this EIV problem. However, its application in this context is flawed as TLS implicitly assumes that the errors between neighborhood pixels are independent. In this paper, a new optical flow estimation method (EIVM) is formulated to properly treat the EIV problem in optical flow. EIVM is based on Sprent's (1966) procedure which allows the incorporation of a general EIV model in the estimation process. In EIVM, the neighborhood size acts as a smoothing parameter. Due to the weights in the EIVM objective function, the effect of changing the neighborhood size is more complex than in other local model methods such as Lucas and Kanade (1981). These weights, which are functions of the flow estimate, can alter the effective size and orientation of the neighborhood. In this paper, we also present a data-driven method for choosing the neighborhood size based on Stein's unbiased risk estimators (SURE).