混合优化,以有效地修剪随机生成的神经网络
Ji Xia1, Huanfei Ma1
1School of Mathematical Sciences, Soochow University, Suzhou 215001, People's Republic of China.
Chaos (Woodbury, N.Y.)
|July 21, 2025
概括
本研究引入了一种混合规范化方法,以压缩随机生成的神经网络,如储库计算 (RC) 和极端学习机器 (ELM),减少神经元数量,同时保持动态系统建模的性能.
科学领域:
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
背景情况:
- 随机生成的神经网络 (RGNN),包括储库计算 (RC) 和极端学习机器 (ELM),通过固定随机权重提供简化训练.
- 然而,由于神经元数量过多,RGNN经常表现出计算冗余性和高硬件要求.
研究的目的:
- 开发一个混合规范化框架来压缩RGNNs.
- 为了平衡网络大小的减少与预测性性能维护.
- 为资源有限的环境中提供动态系统建模的高效解决方案.
主要方法:
- 提出了一个混合规范化框架,将L1和L2优化策略结合起来.
- 应用框架来压缩RGNN用于动态系统建模.
- 在经典混乱系统上进行模拟.
- 用各种修剪策略进行比较实验.
主要成果:
- 优化的网络保留了神经元的核心子集,其预测性能与原始网络相比相当.
- 混合规范化方法在平衡网络紧性和稳定性方面表现优于其他方法.
- 与其他替代方法相比,提出的方法的普遍性更强.
结论:
- 混合规范化框架有效地实现了RGNNs中的显著网络大小压缩.
- 这种方法为动态系统建模提供了有效的解决方案,特别是在计算资源有限的场景中或用于物理实现.
相关概念视频
Survival Tree
163
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
163
Neuroplasticity
786
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
786
Randomized Experiments
7.2K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
7.2K
Neural Regulation
40.1K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
40.1K
Neural Circuits
1.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.6K
Reducing Line Loss
195
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
195
