在模拟硬件上的可转移学习
Sri Krishna Vadlamani1, Dirk Englund1, Ryan Hamerly1,2
1Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
Science advances
|July 12, 2023
概括
新的训练方法为光子电路创建了强大的模拟神经网络 (NN). 这些错误感知技术克服了硬件缺陷,使得高效,高性能的NN即使存在重大制造错误.
科学领域:
- 光子学 是一个光子学.
- 人工智能的人工智能
- 硬件工程 硬件工程
背景情况:
- 模拟神经网络 (NN) 可以节省大量的能量和时间,但容易受到静态制造错误的影响.
- 目前用于光子NN加速器的训练方法不能产生适应硬件缺陷的网络.
- 由于需要再培训,组件需求或硬件开销,现有的错误纠正方法对于大规模部署是不切实际的.
研究的目的:
- 为模拟神经网络开发新的训练技术,以提高对静态硬件错误的稳定性.
- 为了使光子NN加速器在现实世界,容易出错的环境中实际部署.
- 为了解决模拟NN硬件中现有的错误纠正策略的局限性.
主要方法:
- 引入可编程光子干扰仪电路的一次性错误识别训练方法.
- 开发了培训NNs的技术,这些技术本质上是对静态制造变化的坚固的.
- 证明训练有素网络可转移到任意光子NN硬件,但存在重大错误.
主要成果:
- 实现了强大的神经网络 (NN),与理想硬件的性能匹配.
- 成功将训练有素的NN转移到有缺陷的光子硬件中,其误差高达当前制造公差的五倍.
- 克服了需要个人再培训,严格的组件质量或与错误纠正相关的硬件开销的需求.
结论:
- 一次性错误识别训练是创建光子平台上强大的模拟神经网络的可行解决方案.
- 这种方法显著提升了能源效率高的模拟NN加速器的实际应用.
- 开发的技术为在边缘计算和其他苛刻的应用中部署高度可靠的光子NN铺平了道路.
相关概念视频
01:26Higher Mental Functions of Brain: Learning and Memory
881
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
881
01:27Associative Learning
450
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
450
01:12Observational Learning
213
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
213
01:22Multi-input and Multi-variable systems
132
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
132
01:19Electro-mechanical Systems
1.0K
Electromechanical systems are intricate configurations that effectively combine electrical and mechanical elements to achieve a desired outcome. Central to many of these systems is the DC motor, a device that converts electrical energy into mechanical motion, enabling various applications ranging from simple fans to complex robotic mechanisms.
A key component of the DC motor is the armature, a rotating circuit positioned within a magnetic field. As an electric current passes through the...
A key component of the DC motor is the armature, a rotating circuit positioned within a magnetic field. As an electric current passes through the...
1.0K
01:22Purposive Learning
145
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
145