Related Experiment Video
Updated: Jul 9, 2026
13:44
Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy
Published on: August 8, 2011
14.0K
An Embedded Framework for Fully Autonomous Object Manipulation in Robotic-Empowered Assisted Living
Giovanni Mezzina1, Daniela De Venuto1
1Department of Electrical and Information Engineering, Politecnico di Bari, 70125 Bari, Italy.
Sensors (Basel, Switzerland)
|January 8, 2023
Summary
This study introduces the MONOCULAR framework, enabling humanoid robots to manipulate objects using RGB and depth sensors. It achieves high success rates for autonomous object retrieval and delivery in real-world scenarios.
Area of Science:
- Robotics
- Human-Robot Interaction
- Computer Vision
Background:
- Current social robots lack object manipulation skills, limiting their utility beyond verbal interaction.
- Integrating manipulation capabilities is crucial for advancing social robot applications in diverse environments.
Purpose of the Study:
- To present the MONOCULAR (eMbeddable autONomous ObjeCt manipULAtion Routines) framework for enabling object manipulation in social robots.
- To enhance social robot functionality through sensor fusion for autonomous object localization and grasping.
Main Methods:
- Utilized a fusion of RGB cameras and a 3D depth sensor for object localization and manipulation.
- Employed an embedded You Only Look Once (YOLO) object detector for item identification and an edge-detecting algorithm for shelf height estimation.
- Developed dynamic approach and manipulation routines optimized for central or lateral object placement, adapting to real-time sensor data.
Main Results:
- Achieved a 96% grabbing success rate for laterally placed objects and 97% for centrally placed objects.
- Demonstrated the framework's adaptability across various shelf heights and its effectiveness in a drug delivery case study with the Pepper robot.
- Validated the autonomous and privacy-preserving nature of the MONOCULAR procedures.
Conclusions:
- The MONOCULAR framework successfully equips social robots with essential object manipulation capabilities.
- This advancement significantly broadens the potential applications of humanoid robots in domestic, healthcare, and industrial settings.
- The framework offers a robust and adaptable solution for autonomous object handling, enhancing robot utility and interaction.
Related Concept Videos
01:24One-Degree-of-Freedom System
In mechanical engineering, one-degree-of-freedom systems form the basis of a wide range of electrical and mechanical components. Using these models, engineers can predict the behavior of various parts in a larger system, which gives them insight into how different forces interact with each other.
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
01:28Automatic Processing and Automatic Social Behavior
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...