Geometric video approximation using weighted matching pursuit

In recent years, works on geometric multidimensional signal representations have established a close relation with signal expansions on redundant dictionaries. For this purpose, matching pursuits (MP) have shown to be an interesting tool. Recently, most important limitations of MP have been underlined, and alternative algorithms like weighted-MP have been proposed. This work explores the use of weighted-MP as a new framework for motion-adaptive geometric video approximations. We study a novel algorithm to decompose video sequences in terms of few, salient video components that jointly represent the geometric and motion content of a scene. Experimental coding results on highly geometric content reflect how the proposed paradigm exploits spatio-temporal video geometry. Two-dimensional weighted-MP improves the representation compared to those based on 2-D MP. Furthermore, the extracted video components represent relevant visual structures with high saliency. In an example application, such components are effectively used as video descriptors for the joint audio-video analysis of multimedia sequences.