学习基于模拟的视觉策略,用于在现实世界中在看不见的洞中
Liang Xie1, Hongxiang Yu1, Kechun Xu1
1College of Control Science and Engineering, Zhejiang University, Zhejiang, China.
The Review of scientific instruments
|October 9, 2023
概括
这项研究介绍了一种基于学习的视觉入洞系统,可以将其推广到新的形状,最小的模拟到真实转移成本. 它使用一种新的框架快速适应看不见的物体,在现实世界任务中实现高成功率.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 机器人操纵,特别是入洞任务,需要精确的控制和适应变化的能力.
- 当前的方法往往难以将其推广到未见的形状,并且需要大量的现实世界训练数据.
- 弥合模拟和现实世界性能 (模拟到现实转移) 之间的差距仍然是机器人技术的一个重大挑战.
研究的目的:
- 开发一种基于学习的视觉钉入孔系统,能够适应任意的未见形状,并且具有最小的模拟到真实成本.
- 为了提高适应性,将感官运动政策的概括与感知和控制模块脱.
- 为了实现高效的培训和快速部署在现实世界的场景.
主要方法:
- 提出了一个包含分段网络 (SN),虚拟传感器网络 (VSN) 和控制器网络 (CN) 的框架.
- 该VSN受过训练,可以从细分图像中测量对象姿势,从而实现形状无关的姿势测量.
- 使用CN训练了一种通用的挂政策,只有SN为现实世界的转移进行了微调,最大限度地降低了适应成本.
主要成果:
- 该系统证明了成功地适应未见的形状,最小的sim-to-real转移成本.
- 通过使用电动汽车充电系统,在2-3秒内实现了10/10的成功率.
- 这种方法只需要数百个自动标记的样本进行SN传输,大大减少了数据收集工作.
结论:
- 拟议的基于学习的视觉入洞框架有效地解决了机器人操纵任务的模拟到真实转移问题.
- 分离感知和控制模块允许快速适应新型形状,最小微调.
- 该系统在复杂的操纵任务中显示出实际可行性,其在自动电动汽车充电中的高成功率证明了这一点.
相关概念视频
Collisions in Multiple Dimensions: Problem Solving
4.2K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
4.2K
Prismatic Beams: Problem Solving
119
In the design of a supported timber beam subjected to a distributed load, both the beam's physical dimensions and the timber's characteristics, such as its grade and species, are critical. These factors determine the allowable stress values, which are crucial for calculating the necessary beam depth to ensure structural integrity and safety.
The design begins with analyzing the beam as a free body to identify moments and force balances, thereby determining support reactions. Next, the...
The design begins with analyzing the beam as a free body to identify moments and force balances, thereby determining support reactions. Next, the...
119
Frames: Problem Solving II
238
Consider a hydraulic hoist supporting a load of 1 kN. Assuming a simplified schematic representation of this frame structure, the force acting on BD and BF members can be determined.
238
Statically Indeterminate Problem Solving
432
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
432
Frictional Forces on Screws
1.2K
Screws are characterized by a helical ridge known as a thread wrapped around a cylindrical shaft. They are commonly used as fasteners to hold objects together or to transmit power and motion in machines. One type of screw that is particularly useful for transmitting power is the square-threaded screw.
A jack with a square-threaded screw is a mechanical device used to lift heavy loads by applying a force at its handle. When the force is applied, the screw turns, raising the load. The screw can...
A jack with a square-threaded screw is a mechanical device used to lift heavy loads by applying a force at its handle. When the force is applied, the screw turns, raising the load. The screw can...
1.2K
Frames: Problem Solving I
496
Consider a jib crane with an external load suspended from the pulley. The dimensions of the crane members are shown in the figure. A systematic analysis of the frame structure is required to determine the reaction forces at the pin joints, assuming that the pulleys are frictionless.
496


