相关实验视频
Updated: Jun 9, 2026

13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
9.2K
许多深度网络的训练过程探索了相同的低维多元
Jialin Mao1, Itay Griniasty2, Han Kheng Teoh2
1Applied Mathematics and Computational Sciences, University of Pennsylvania, Philadelphia, PA 19104.
概括
深度网络培训探索一个低维的多元组,无论架构或优化. 不同的网络架构显示出不同的路径,但相似地融合,揭示出一种普遍的训练动态.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 信息几何学信息几何学
背景情况:
- 深度神经网络 (DNN) 是人工智能的强大工具.
- 了解DNN培训的内部动态对于提高模型性能和可靠性至关重要.
- 现有的方法往往难以捕捉培训过程的复杂性和高维度性质.
研究的目的:
- 在培训期间应用信息几何技术来分析深度网络预测轨迹.
- 通过DNN预测探索空间的内在维度.
- 确定各种训练因素对探索这个空间的影响.
主要方法:
- 开发用于分析高维概率模型的信息几何方法.
- 检查各种架构和训练配置的深度网络的预测轨迹.
- 在各种优化,规范化,数据增强和重量初始化策略下对多重探索进行比较分析.
主要成果:
- 深度网络训练始终在预测空间中有效地探索一个低维的多元组.
- 这种共享的多样性在不同的架构,大小和培训方法的网络中观察到.
- 虽然架构表现出可区分的轨迹,但其他因素,如大小和初始化对多元融合的影响最小.
结论:
- 深度网络的训练,无论具体的配置如何,都汇聚到一个共同的低维分组.
- 网络架构影响了轨迹,但并没有影响到探索的基本多元.
- 这一发现为深度学习培训动态提供了统一的视角,简化了我们对复杂模型的理解.
相关概念视频
Collisions in Multiple Dimensions: Introduction
It is far more common for collisions to occur in two dimensions; that is, the initial velocity vectors are neither parallel nor antiparallel to each other. Let's see what complications arise from this. The first idea is that momentum is a vector. Like all vectors, it can be expressed as a sum of perpendicular components (usually, though not always, an x-component and a y-component, and a z-component if necessary). Thus, when the statement of conservation of momentum is written for a problem,...
Collisions in Multiple Dimensions: Problem Solving
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
Multi-input and Multi-variable systems
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
Sequence Networks of Rotating Machines
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Introduction to Learning
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
Vectors in 2D: Problem Solving
A plane traveling due north at 180 km/h in still air was found to be 80 km off-course after 30 minutes, deviating approximately 5 degrees east of north. This deviation means the influence of a crosswind alters the plane’s intended trajectory. The actual ground path formed a diagonal, suggesting that the aircraft’s effective ground speed was reduced to 160 km/h and directed slightly to the east due to the wind.By analyzing the displacement from the intended path, the velocity contributed by the...

