Temporally sequenced intelligent block-matching and motion-segmentation using locally coupled networks

Motion-based segmentation is a very important capability for computer vision and video analysis. It depends fundamentally on the system's ability to estimate optic flow using temporally proximate image frames. This is often done using block-matching. However, block-matching is sensitive to the presence of observational noise, which is inevitable in real images. Also, images often include regions of homogeneous intensity, where block-matching is problematic. A better method in this case is to estimate motion at the region level. In the approach described in this paper, we have attempted to address the noise-sensitivity and texture-insufficiency problems using a two-pathway system. The pixel-level pathway is a multilayer pulse-coupled neural network (PCNN)-like locally coupled network used to correct outliers in the block-matching motion estimates and produce improved estimates in regions with sufficient texture. In contrast, the region-level pathway is used to estimate the motion for regions with little intensity variation. In this pathway, a PCNN network first partitions intensity images into homogeneous regions, and a motion vector is then determined for the whole region. The optic flows from both pathways are fused together based on the estimated intensity variation. The fused optic flow is then segmented by a one-layer PCNN network. Results on synthetic and real images are presented to demonstrate that the accuracy of segmentation is improved significantly by taking advantage of the complementary strengths and weaknesses of the two pathways.