有限时间的Lyapunov 深度神经网络的指数
L Storm1, H Linander2, J Bec3,4
1Department of Physics, University of Gothenburg, 41296 Gothenburg, Sweden.
Physical review letters
|February 16, 2024
概括
深度神经网络通过在输入空间中创建几何结构来学习. 这些结构,由Lyapunov指数可视化,指导网络如何分类数据.
科学领域:
- 计算神经科学是一种计算神经科学.
- 机器学习理论机器学习理论
- 动态系统理论 动态系统理论
背景情况:
- 深度神经网络 (DNN) 表现出复杂的行为.
- 了解他们的内部决策过程至关重要.
- 输入干扰可以显著改变DNN输出.
研究的目的:
- 分析小输入扰动对DNN输出的影响.
- 为了在DNN和动态系统之间进行类比.
- 为了可视化DNN学到的几何结构.
主要方法:
- 计算DNN的有限时间利亚普诺夫指数.
- 在输入空间中探索这些指数的几何解释.
- 识别与分类界限相关的结构.
主要成果:
- 最大Lyapunov指数在输入空间中形成几何结构.
- 这些结构类似于动态系统中发现的连贯结构.
- 高正指数的山脊划分与不同类别相关的区域.
结论:
- 在学习过程中,DNN在输入空间中构建特定的几何形状.
- 利亚普诺夫指数提供了一个可视化和理解这些学习几何的工具.
- 这种方法揭示了DNN学习的基本机制.
相关概念视频
Long-term Potentiation
55.2K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.2K
Exponential and Sinusoidal Signals
264
The exponential function is crucial for characterizing waveforms that rise and decay rapidly. This continuous-time exponential function is defined using exponential terms with constants α and A. When both constants are real, the function is represented as,
264
Linear Approximation in Frequency Domain
91
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
91
Exponential Fourier series
202
In audio signal processing, the exponential Fourier series plays a crucial role in sound synthesis, allowing complex sounds to be broken down into simpler sinusoidal components. This decomposition process is fundamental in analyzing and reconstructing musical notes and other audio signals. The exponential Fourier series expresses periodic signals as the sum of complex exponentials at both positive and negative harmonic frequencies, providing a powerful tool for signal analysis.
Euler's identity...
Euler's identity...
202
Linear time-invariant Systems
258
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
258
Discrete-Time Fourier Series
267
The Discrete-Time Fourier Series (DTFS) is a fundamental concept in signal processing, serving as the discrete-time counterpart to the continuous-time Fourier series. It allows for the representation and analysis of discrete-time periodic signals in terms of their frequency components. Unlike its continuous counterpart, which utilizes integrals, the calculation of DTFS expansion coefficients involves summations due to the discrete nature of the signal.
For a discrete-time periodic signal x[n]...
For a discrete-time periodic signal x[n]...
267


