Object-grounded embodied picking for e-commerce warehouse fulfillment: a foveated diffusion policy for operational...

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Embodied picking in e-commerce is challenged by visual complexities like clutter and varied packaging.
• Existing visuomotor policies lack explicit object grounding, leading to reliance on irrelevant contextual cues.
• This hinders the effectiveness and robustness of robotic systems in real-world fulfillment scenarios.

Purpose of the Study:

• To introduce the Foveated Diffusion Policy (FDP) for integrating object-centric visual grounding into diffusion-based action generation.
• To enhance the robustness and accuracy of visuomotor policies for embodied picking tasks.
• To address the limitations of current methods in handling complex visual environments.

Main Methods:

• Developed a panorama-fovea dual-stream visual encoder to capture global context and high-resolution object details.
• Integrated object tokens into a diffusion denoiser using object-guided cross-attention for grounded action generation.
• Evaluated the FDP on six simulated manipulation tasks across ten benchmark datasets using human demonstrations.

Main Results:

• FDP achieved average success rates of 81.0% (proficient-human) and 76.8% (multi-human demonstrations).
• Demonstrated superior performance on object-centric tasks like Push-T (86.7%) and Lift (100.0%).
• Showcased improved robustness against localization perturbations and reduced inference latency for closed-loop control.

Conclusions:

• Explicit object grounding via FDP significantly improves the robustness of visuomotor policies in simulated warehouse environments.
• The FDP approach is computationally practical and enhances performance in challenging, cluttered scenarios.
• This method offers a promising direction for advancing automated e-commerce fulfillment through more capable robotic systems.