3D Rigid Motion Segmentation with Mixed and Unknown Number of Models

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Traditional motion segmentation methods often rely on a single model (homography or fundamental matrix), which struggles with general scenes or degenerate cases.
• Existing approaches face difficulties when categorizing video sequences into simple or complex motion models, leading to performance limitations.

Purpose of the Study:

• To develop a robust motion segmentation framework that overcomes the limitations of single-model approaches.
• To introduce a synergistic approach combining multiple motion models for enhanced video analysis.
• To address the open problem of model selection for estimating the number of independently moving objects.

Main Methods:

• Proposed a multi-model spectral clustering framework integrating both homography and fundamental matrix models.
• Developed novel model selection criteria balancing data fidelity and model complexity.
• Evaluated the framework on existing datasets and a new, challenging dataset adapted from the KITTI benchmark.

Main Results:

• Achieved state-of-the-art performance on both motion segmentation and model selection tasks across multiple datasets.
• Demonstrated substantial performance improvements by synergistically combining multiple motion models.
• Validated the framework's effectiveness on a new dataset featuring realistic challenges like strong perspectives and forward translations.

Conclusions:

• The proposed multi-model spectral clustering framework offers a significant advancement in video motion segmentation.
• Synergistic integration of homography and fundamental matrix models enhances robustness and accuracy.
• The developed model selection criteria effectively address a key challenge in unsupervised motion analysis.