Collapse and collision aware grasping for cluttered shelf picking

In modern smart factories, automated shelf picking must deliver high throughput, flexibility, and safe human-robot collaboration. In these environments, efficient and safe retrieval of stacked objects is a significant challenge due to complex spatial dependencies and structural inter-dependencies. Traditional vision-based methods excel at object localization but often lack the physical reasoning required to predict the consequences of extraction, leading to unintended collisions and collapses. This paper proposes a collapse and collision-aware grasp planner that integrates dynamic physics simulations for robotic decision-making. Using a single image and depth map, an approximate 3D representation of the scene is reconstructed in a simulation environment, enabling the robot to evaluate different retrieval strategies before execution. Two approaches: 1) heuristic-based and 2) physics-based are proposed for both single-box extraction and shelf clearance tasks. Extensive real-world experiments on structured and unstructured box stacks, along with validation using datasets from existing databases, show that our physics-aware method significantly improves efficiency and success rates compared to baseline heuristics. A video demonstrating the real-world implementation of our proposed system is available at: https://youtu.be/GBWMiNIHUlU.