使用模拟和临时内存权重进行芯片上训练的增强规范化
Raghav Singhal1, Vivek Saraswat1, Shreyas Deshmukh1
1Department of Electrical Engineering, Indian Institute of Technology Bombay, Mumbai, India.
概括
临时模拟记忆细胞加速了人工神经网络 (ANN) 的芯片内学习. 这种方法提供了类似于正规化的好处,在手写数字识别任务中显著减少验证错误.
科学领域:
- 计算机工程 计算机工程
- 人工智能的人工智能
- 材料科学 材料科学 材料科学
背景情况:
- 内存计算加速了人工神经网络 (ANN) 的训练和推理.
- 边缘设备的芯片内学习面临由于频繁的重量更新而面临的挑战.
研究的目的:
- 提出一个模拟和临时的芯片内存 (ATOM) 单元,用于在芯片上训练时实现ANN权重.
- 评估可控制的保留时间表对学习成绩的影响.
主要方法:
- 在GlobalFoundries的45nmRFSOI技术中制造了一个ATOM电池.
- 测量了ATOM单元的读写时间表.
- 在MNIST数据集上训练了一个完全连接的神经网络,评估临时记忆和体重衰减的影响.
主要成果:
- 暂时记忆诱导的体重下降提供了规范化好处,减少了约33%的验证错误 (从3.6%降至2.4%).
- 可控制的衰变时间表进一步减少了~26%的验证错误.
- 证明了临时模拟内存对于高性能芯片上ANN学习的实用性.
结论:
- 临时模拟记忆细胞对芯片上学习是有效的,它补充了用于存储和推断的非易失性记忆.
- 使用临时内存的算法电路编码设计使得芯片上高效和高性能ANN成为可能.
- 在ATOM细胞中可控制的保留时间表为优化边缘设备上的ANN训练提供了一种新的方法.
相关概念视频
Understanding Memory
553
Memory is the retention of information or experiences over time, facilitated through three main processes: encoding, storage, and retrieval. Encoding is the process of inputting information into the memory system. For instance, when listening to a lecture, watching a play, reading a book, or having a conversation, the brain is actively encoding information. This initial stage involves transforming sensory input into a form that can be processed and stored by the brain. Various factors, such as...
553
Long-term Potentiation
2.8K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.8K
System of Memory
6.3K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
6.3K
Multi-input and Multi-variable systems
129
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
129
Long-Term Memory
209
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
209
Implicit Memories
154
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
154


