Multi-view object pose distribution tracking for pre-grasp planning on mobile robots

The ability to track the 6D pose distribution of an object while a mobile manipulator is still approaching it can enable the robot to pre-plan grasps, thereby improving both the time efficiency and robustness of mobile manipulation. However, tracking a 6D object pose distribution on approach can be challenging due to the limited view of the robot camera. In this study, we present a particle filter-based multi-view 6D pose distribution tracking framework that compensates for the limited view of the moving robot camera while it approaches the object by fusing observations from external stationary cameras in the environment. We extend the single-view pose distribution tracking framework (PoseRBPF) to fuse observations from external cameras. We model the object pose posterior as a multi-modal distribution and introduce techniques for fusion, re-sampling, and pose estimation from the tracked distribution to effectively handle noisy and conflicting observations from different cameras. To evaluate our framework, we also contribute a real-world benchmark dataset. Our experiments demonstrate that the proposed framework yields a more accurate quantification of object pose and associated uncertainty than previous research. Finally, we apply our framework for pre-grasp planning on mobile robots, demonstrating its practical utility.