一个具有记忆力的包容性超级网络,用于平行模拟部署全搜索空间架构
Bo Lyu1, Yin Yang2, Yuting Cao2
1Zhejiang Lab, Hangzhou, Zhejiang, China.
概括
本研究介绍了一种用于神经架构搜索 (NAS) 的全包式超级网络,它可以在内存硬件上灵活地部署各种深度学习模型. 拟议的超级网络有效地覆盖了广的搜索空间,在CIFAR10数据集上实现了具有竞争力的准确性,并且具有合理的硬件开销.
科学领域:
- * 计算机工程 计算机工程
- * 人工智能 * 人工智能
- * 材料科学 材料科学
背景情况:
- *基于memristor的神经网络正在推进深度学习的内存处理 (PIM).
- *当前的并行模拟记忆平台在特征图生成方面表现出色,但缺乏针对各种神经网络结构的灵活性.
- *神经架构搜索 (NAS) 创建专业架构,但在异质的记忆硬件上面临部署挑战.
研究的目的:
- * 调查能够克服固定结构内存平台的局限性的内存模拟部署策略.
- * 开发一种灵活和高效的解决方案,用于在内存硬件上部署NAS生成的架构.
- * 提出一个具有记忆力的包容性超级网络,能够覆盖广泛的神经网络架构.
主要方法:
- * 设计了用于DARTS (差分架构搜索) 空间内原始操作的记忆硬件.
- * 开发了一个具有记忆力的全包容性超级网络,可以包含大量的网络架构 (2×10^25).
- *使用CIFAR10数据集对代表性架构 (DARTS-V1,DARTS-V2,PDARTS) 进行了计算模拟,并评估了硬件性能.
主要成果:
- * 在CIFAR10数据集上,具有记忆性的全包容性超级网络实现了有希望的准确性 (89.2%的PDARTS具有8位定量化).
- * 与DARTS完整搜索空间内的所有架构的兼容性得到证明.
- * 硬件模拟表明,与具有可比功率的单个部署相比,资源消耗略有增加 (22%25%的延迟,1.5×面积).
结论:
- * 拟议的memristive全包式超级网络为在并行模拟memristive硬件上部署各种NAS生成架构提供了可行的解决方案.
- * 这种方法在工业应用中提供了灵活性,性能和资源利用之间的合理权衡.
- * 这项工作推动了PIM领域的发展,通过在memristive平台上实现更具适应性和效率的深度学习部署.
相关概念视频
01:17Ampere-Maxwell's Law: Problem-Solving
626
A parallel-plate capacitor with capacitance C, whose plates have area A and separation distance d, is connected to a resistor R and a battery of voltage V. The current starts to flow at t = 0. What is the displacement current between the capacitor plates at time t? From the properties of the capacitor, what is the corresponding real current?
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
626
01:31Ampere's Law: Problem-Solving
3.6K
Ampere's law states that for any closed looped path, the line integral of the magnetic field along the path equals the vacuum permeability times the current enclosed in the loop. If the fingers of the right hand curl along the direction of the integration path, the current in the direction of the thumb is considered positive. The current opposite to the thumb direction is considered negative.
Specific steps need to be considered while calculating the symmetric magnetic field distribution...
Specific steps need to be considered while calculating the symmetric magnetic field distribution...
3.6K
01:25Network Function of a Circuit
286
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
286
01:25Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K