神经网络组合的内在维度 神经网络组合的内在维度
Francesco Tosti Guerra1, Andrea Napoletano2, Andrea Zaccaria2
1Department of Physics, Sapienza University of Rome, 00185 Rome, Italy.
Entropy (Basel, Switzerland)
|April 26, 2025
概括
随机网络初始化显著增加神经网络组合的变化,而不是数据扭曲或丢失. 了解这些训练影响可以优化模型性能和参数空间探索.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 神经网络在训练过程中表现出复杂的集体行为.
- 这些行为通常可视化在不断演变的数学结构上,称为多元体.
- 本质维度量化了组合变异性和培训策略影响.
研究的目的:
- 分析神经网络组合的集体行为.
- 量化各种培训策略对参数空间探索的影响.
- 了解训练选择如何影响模型准确性.
主要方法:
- 在复杂的多元体上使用内在维度来表征神经网络组合.
- 量化受培训选择影响的参数空间覆盖面.
- 调查随机初始化,数据扭曲,中断和批量混的影响.
主要成果:
- 随机初始化是神经网络组合中变化的主要驱动因素.
- 内在维度与参数空间覆盖面和网络可变性相关.
- 分析了培训策略的组合及其对预测准确性的影响.
结论:
- 训练选择,特别是随机初始化,显著影响神经网络组合动态.
- 较高的内在维度表明更大的可变性和参数空间探索.
- 这项研究突出了培训策略在深度学习中的关键,往往被低估的作用.
相关概念视频
Neural Circuits
919
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
919
Neural Regulation
39.0K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.0K
Neuroplasticity
245
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
245


