Shape Tracking with Occlusions via Coarse-to-Fine Region-Based Sobolev Descent.

Area of Science:

• Computer Vision
• Geometric Deep Learning
• 3D Reconstruction

Background:

• Object shape tracking from video is crucial for various applications.
• Existing joint shape and appearance models struggle with self-occlusions and dis-occlusions.
• Inaccurate shape detection results from the inability to adapt to new visual information.

Purpose of the Study:

• To develop a robust method for tracking object shapes in videos, specifically addressing challenges posed by self-occlusions and dis-occlusions.
• To improve the accuracy and adaptability of shape tracking models.

Main Methods:

• A joint shape and appearance model is propagated across video frames to determine object shape.
• Self-occlusions and dis-occlusions are explicitly modeled within a joint shape and appearance tracking framework.
• A coupled optimization problem is formulated, integrating self-occlusions and model propagation.
• A coarse-to-fine optimization method is derived using gradient descent on a novel infinite-dimensional Riemannian manifold with a Sobolev metric.

Main Results:

• Experiments demonstrate superior shape accuracy in videos with occlusions, dis-occlusions, complex radiance, and backgrounds.
• The proposed method effectively handles challenging scenarios that hinder traditional tracking approaches.
• The novel Riemannian manifold and Sobolev metric facilitate robust model perturbation and tracking.

Conclusions:

• Modeling self-occlusions and dis-occlusions significantly enhances shape tracking accuracy.
• The developed coarse-to-fine optimization strategy on a Riemannian manifold provides an advantageous approach for video tracking.
• This method offers a more reliable solution for object shape determination in dynamic visual environments.