贝叶斯神经网络中的层适应节点选择:统计保证和实施细节
Sanket Jantre1, Shrijita Bhattacharya1, Tapabrata Maiti1
1Department of Statistics and Probability, Michigan State University, United States of America.
概括
我们介绍了一种新的贝叶斯稀疏深度神经网络方法,用于高效的模型构建. 这种方法自动选择节点,减少结构复杂性,提高计算速度,以便更好地预测.
科学领域:
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
- 统计建模 统计建模
背景情况:
- 稀疏的深度神经网络 (DNN) 对于大规模的预测建模至关重要.
- 现有的方法往往侧重于边缘选择,这可能不会降低结构复杂性.
- 修剪过多的节点在推理过程中提供了更大的计算速度.
研究的目的:
- 为DNN中的自动节点选择提出贝叶斯稀疏解决方案.
- 开发一种方法,避免对修剪的临时门规则.
- 在稀疏的DNN中提高计算效率和预测性能.
主要方法:
- 使用spike-and-slab高斯先验来自动选择节点.
- 采用一个变化的贝叶斯方法来克服马尔科夫链蒙特卡洛 (MCMC) 的挑战.
- 建立变化的后部一致性和表征先前的参数.
主要成果:
- 与边缘选择方法相比,证明了更高的计算复杂性.
- 与现有方法相比,实现了类似或更好的预测性能.
- 由理论框架促进的经验验证的层级智能最佳节点恢复.
结论:
- 提出的贝叶斯稀疏DNN方法通过自动节点选择有效地降低了结构复杂性.
- 带有spike-and-slab priors的变量贝叶斯为MCMC提供了一个计算效率高的替代方案.
- 这种方法推进了稀疏网络设计,实现了显著的加快速度并保持了预测准确度.
相关概念视频
Neural Circuits
1.3K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.3K
Neural Regulation
39.5K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.5K
Neuroplasticity
560
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
560
Survival Tree
109
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
109
Randomized Experiments
7.0K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
7.0K


