Learning joint intensity-depth sparse representations.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Scene Understanding

Background:

• Learning representations from multiple data modalities is challenging.
• Existing methods struggle to capture joint sparsity across different signal types.

Purpose of the Study:

• To develop a novel algorithm for learning dictionaries of features from 3D scene intensity and depth data.
• To enable the recovery of generative models where underlying 3D causes produce distinct intensity and depth signals.

Main Methods:

• Proposed a Joint Basis Pursuit (JBP) algorithm using conic programming for related sparse feature detection.
• Integrated JBP into a two-step dictionary learning framework.
• Developed a recovery error bound for JBP and compared it numerically to group lasso.

Main Results:

• JBP successfully identifies joint sparsity models with distinct atoms and coefficients for intensity and depth.
• The learning algorithm converges to related features like depth/intensity edges and texture/slant pairs.
• JBP demonstrates superior performance over state-of-the-art methods in depth inpainting tasks.

Conclusions:

• The proposed JBP algorithm effectively learns related sparse features from multi-modal 3D scene data.
• This approach advances generative model recovery and improves 3D depth inpainting accuracy.