Object-centric diffusion policies for real-world robotic-arm imitation learning

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Imitation learning in complex environments is difficult due to perception grounding and action modeling challenges.
• Existing methods using pixel-level features or latent variables struggle with cluttered scenes and brittle attention.
• Unstructured environments require robust visual representations for effective robotic control.

Purpose of the Study:

• To present a novel framework integrating detector-based visual representations with conditional diffusion modeling (DINO + CDP) for real-world robotic imitation learning.
• To systematically quantify the impact of scene complexity on robotic policy performance.
• To demonstrate the effectiveness of grounding action generation in object-level features for improved robustness.

Main Methods:

• Utilized a DINO object detection transformer for spatially-grounded object-query embeddings.
• Employed conditional diffusion modeling (CDP) for policy generation, conditioned on object-query embeddings.
• Quantified scene complexity using image entropy and compared performance against pixel-centric models.

Main Results:

• DINO + CDP mitigated performance degradation caused by high scene complexity (image entropy).
• The framework demonstrated improved task success rates and smoother trajectories in real-world robotic manipulation.
• Object-query-conditioned diffusion showed superior robustness to high-entropy visual inputs compared to baseline methods.

Conclusions:

• DINO + CDP offers a scalable pathway for imitation learning in challenging domains like agriculture.
• Grounding action generation in stable object-level features enhances robotic policy performance in complex environments.
• The approach overcomes limitations of pixel-centric models by leveraging structured visual information.