Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Nonconscious Mimicry01:13

Nonconscious Mimicry

4.7K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.7K
Modeling and Similitude01:12

Modeling and Similitude

373
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
373
Observational Learning01:12

Observational Learning

372
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
372
Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

198
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
198
Stereotype Content Model02:16

Stereotype Content Model

14.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.9K
Automatic Processing and Automatic Social Behavior01:28

Automatic Processing and Automatic Social Behavior

22
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
22

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Interoceptive autonomic regulation in typical development and autism spectrum disorder: A computational model integrating multiple physiological systems.

PloS one·2026
Same author

Embodied predictive processing: A unifying generative principle for cognitive development.

Advances in child development and behavior·2026
Same author

Mother-child autonomic nervous system interaction as an indication of parental stress: 24-hour cross recurrence plot analysis.

PloS one·2025
Same author

The rubber tool illusion reveals how body image modifies body schema.

Journal of experimental psychology. Human perception and performance·2025
Same author

Success-efficient/failure-safe strategy for hierarchical reinforcement motor learning.

PLoS computational biology·2025
Same author

Machine learning model for reproducing subjective sensations and alleviating sound-induced stress in individuals with developmental disorders.

Frontiers in psychiatry·2025

Related Experiment Video

Updated: Oct 11, 2025

Robotic Mirror Therapy System for Functional Recovery of Hemiplegic Arms
10:32

Robotic Mirror Therapy System for Functional Recovery of Hemiplegic Arms

Published on: August 15, 2016

15.6K

Imitation and mirror systems in robots through Deep Modality Blending Networks.

M Yunus Seker1, Alper Ahmetoglu1, Yukie Nagai2

  • 1Bogazici University, Bebek, Istanbul, 34342, Turkey.

Neural Networks : the Official Journal of the International Neural Network Society
|November 28, 2021
PubMed
Summary

This study introduces deep modality blending networks (DMBN) for robots to understand actions and imitate behaviors by creating a shared latent space from multi-modal experiences. DMBN enables robust mirror learning, outperforming vision-only models, by integrating proprioceptive and visual data.

Keywords:
Imitation learningMultimodal learningRepresentation learningRobot learning

More Related Videos

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.3K
Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation
06:53

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Published on: March 1, 2017

13.4K

Related Experiment Videos

Last Updated: Oct 11, 2025

Robotic Mirror Therapy System for Functional Recovery of Hemiplegic Arms
10:32

Robotic Mirror Therapy System for Functional Recovery of Hemiplegic Arms

Published on: August 15, 2016

15.6K
Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.3K
Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation
06:53

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Published on: March 1, 2017

13.4K

Area of Science:

  • Robotics and Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Robots learning to interact with environments enhances manipulation, action understanding, and imitation.
  • Biological systems, like primates with mirror neurons, demonstrate multi-modal action understanding.
  • Enabling robots to leverage interaction experience for understanding others' actions remains a challenge.

Purpose of the Study:

  • To propose a novel method, deep modality blending networks (DMBN), for creating a common latent space from multi-modal robot experiences.
  • To demonstrate DMBN's capability in facilitating action recognition and enabling anatomical and effect-based imitation.
  • To establish a computational model for mirror neuron-like capabilities and a robust machine learning architecture for multi-modal temporal data.

Main Methods:

  • Developed deep modality blending networks (DMBN) using a stochastic weighting mechanism to blend multi-modal signals into a common latent space.
  • Utilized conditional neural processes allowing conditioning on any sensory/motor value for one-shot generation of complete multi-modal trajectories.
  • Conducted simulation experiments with an arm-gripper robot and RGB camera, comparing DMBN with multi-modal variational autoencoders.

Main Results:

  • DMBN accurately predicted missing modalities (camera or joint angles) and outperformed multi-modal variational autoencoders in long-horizon trajectory predictions.
  • The system generated corresponding image and joint angle sequences for anatomical or effect-based imitation based on desired images.
  • Mirror learning was achieved in all DMBN scenarios, whereas vision-only models failed in half, highlighting the necessity of proprioceptive experience.

Conclusions:

  • DMBN provides a novel approach to action recognition and imitation by creating a shared latent space from multi-modal robot interaction data.
  • The proposed architecture effectively enables mirror neuron-like behavior without pixel-based matching, relying on the blended latent space.
  • DMBN serves as a powerful machine learning architecture for high-dimensional, multi-modal temporal data, demonstrating robust retrieval with partial information.