Probabilistic ToF and stereo data fusion based on mixed pixels measurement models

Area of Science:

• Computer Vision
• Robotics
• 3D Sensing

Background:

• Depth measurement using Time-of-Flight (ToF) cameras is susceptible to artifacts like depth discontinuities caused by the mixed pixel effect.
• Integrating ToF data with stereo vision data offers potential for improved depth accuracy but requires effective fusion strategies.

Purpose of the Study:

• To propose a novel method for fusing depth data from ToF cameras and stereo pairs.
• To develop a ToF depth measurement model that accounts for mixed pixel artifacts.
• To enhance the accuracy and computational efficiency of 3D data fusion techniques.

Main Methods:

• A depth measurement model for ToF cameras was developed, specifically addressing mixed pixel artifacts and depth discontinuities.
• The proposed model was integrated into both Maximum Likelihood (ML) and Markov Random Field (MRF) frameworks for data fusion.
• A site-dependent range value approach was implemented within the MAP-MRF framework to improve accuracy and reduce computational load.
• An extension to Loopy Belief Propagation was introduced for optimizing the site-dependent global cost function.

Main Results:

• Experimental validation confirmed the accuracy of the proposed ToF measurement model.
• The fusion techniques, particularly the MAP-MRF approach with site-dependent ranges, demonstrated significant effectiveness in combining ToF and stereo data.
• The extended Loopy Belief Propagation showed promise for optimizing complex cost functions in related contexts.

Conclusions:

• The proposed ToF measurement model effectively handles depth discontinuities arising from mixed pixels.
• The developed data fusion methods, leveraging ML and MAP-MRF frameworks, significantly improve 3D depth perception accuracy.
• The MAP-MRF approach with site-dependent ranges offers a computationally efficient and accurate solution for ToF and stereo data fusion.