递归最小方程算法的可扩展实现,用于训练尖端神经网络
Benjamin J Arthur1, Christopher M Kim1,2, Susu Chen1
1Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, United States.
Frontiers in neuroinformatics
|July 13, 2023
概括
我们开发了使用优化的CPU和GPU实现来训练尖端神经网络的更快算法. 这加速了对大脑计算的研究,并允许实时模型训练与实验一起进行.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 尖端循环神经网络 (SNN) 越来越多地用于模拟神经计算.
- 在复杂数据上训练大规模的SNNs是计算密集且耗时的.
- 有效的训练算法对于推进神经科学研究至关重要.
研究的目的:
- 为训练SNN提供递归最小方程算法的优化CPU和GPU实现.
- 显著减少训练大型SNN模型所需的时间和资源.
- 为了实现更交互和实时的计算神经科学研究.
主要方法:
- 开发了递归最小平方算法的优化CPU和GPU实现.
- 在数百万个神经元和数十亿个突触的大规模SNN上测试了实现.
- 通过训练一个网络来复制来自小鼠决策任务的广泛神经元记录来验证该方法.
主要成果:
- 该GPU实现的训练速度大约是未经优化的CPU参考速度的1000倍.
- 成功训练了一个大型网络 (>66,000个神经元),在不到一个小时内复制小鼠的大脑活动.
- 证明了该方法对高达100万个神经元的网络的可扩展性.
结论:
- 优化的递归最小平方算法为训练SNN提供了显著的加快速度.
- 允许对复杂的神经动态和多区域计算进行交互式in-silico研究.
- 促进实时模型训练与实体实验并发,弥合计算和实验神经科学.
相关概念视频
Calibration Curves: Linear Least Squares
1.4K
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...
1.4K
Neural Regulation
39.6K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.6K


