Related Experiment Video
Updated: Sep 18, 2025
11:18
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
Published on: March 2, 2015
10.4K
Controlling diverse robots by inferring Jacobian fields with deep networks
Sizhe Lester Li1, Annan Zhang2, Boyuan Chen2
1Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA. sizheli@mit.edu.
Nature
|June 25, 2025
Summary
This study introduces a novel deep neural network method for robot control using only a single camera. This approach maps video streams to a robot
Area of Science:
- Robotics
- Artificial Intelligence
- Computer Vision
Background:
- Controlling biologically inspired robots is challenging due to their complex structures, diverse materials, and lack of sensing.
- Existing control software struggles with soft, multi-material robots that may change properties over time.
Purpose of the Study:
- To develop a universal method for controlling diverse robotic systems using only visual input.
- To overcome limitations of traditional robot modeling and control for unconventional hardware.
Main Methods:
- Utilized deep neural networks to map robot video streams to their visuomotor Jacobian fields.
- Trained the system without expert intervention by observing random actuator commands.
Main Results:
- Enabled accurate closed-loop control of robots using a single camera, regardless of material or actuation.
- Demonstrated the method on a variety of robot manipulators with different characteristics.
- Successfully recovered the causal dynamic structure of each robot.
Conclusions:
- The developed method broadens the design space for robotic systems.
- This approach lowers the barrier to robotic automation by relying solely on camera input.
Related Concept Videos
01:22Multi-input and Multi-variable systems
152
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
152
01:24Sequence Networks of Rotating Machines
150
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
150
01:18Hierarchy of Motor Control
3.7K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
3.7K
01:24One-Degree-of-Freedom System
563
In mechanical engineering, one-degree-of-freedom systems form the basis of a wide range of electrical and mechanical components. Using these models, engineers can predict the behavior of various parts in a larger system, which gives them insight into how different forces interact with each other.
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
563
01:17Open and closed-loop control systems
1.0K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.0K
01:27Transformers in Distribution System
167
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
167