Visual Micro-Pattern Propagation

Statistic observations demonstrate that visual feature patterns or structure patterns recur high-frequently within/across homo/heterogeneous images. Motivated by the interdependencies of visual patterns, we propose visual micro-pattern propagation (VMPP) to facilitate universal visual pattern learning. Especially, we present a graph framework to unify the conventional micro-pattern propagations in spatial, temporal, cross-modal and cross-task domains. A general formulation of pattern propagation named cross-graph model is presented under this framework, and accordingly a factorized version is derived for more efficient computation as well as better understanding. To correlate homo/heterogeneous patterns, in cross-graph we introduce two types of pattern relations from feature-level and structure-level. The structure pattern relation defines second-order visual connections for heterogeneous patterns by measuring first-order visual relations of homogeneous feature patterns. In virtue of the constructed first-/second-order connections, we design feature pattern diffusion and structure pattern diffusion to prop up various pattern propagation cases. To fulfill different pattern diffusions involved, further, we deeply study two fundamental visual problems, multi-task pixel-level prediction and online dual-modal object tracking, and accordingly propose two end-to-end pattern propagation networks by encapsulating and integrating some necessary diffusion modules therein. We conduct extensive experiments by dissecting every diffusion component as well as comparing numerous advanced methods. The experiments validate the effectiveness of our proposed various pattern diffusion ways and meantime report the state-of-the-art results on the two representative visual problems.