对于反向问题,DEs启发的加速展开的线性化的ADMM网络
IEEE transactions on neural networks and learning systems
|April 16, 2024
概括
本研究将展开的线性交替方向乘法方法 (ADMMs) 与微分方程 (DE) 联系起来. 新的形ADMM方案为使用深度网络的反向问题提供了更高的准确性和效率.
科学领域:
- 优化算法 优化算法
- 深度学习 (Deep Learning) 是一种深度学习.
- 应用数学 应用数学 应用数学
背景情况:
- 传统的交替方向乘法方法 (ADMMs) 越来越多地通过连续时间微分方程 (DE) 来理解.
- 展开的深度网络继承了ADMM的代,但缺乏明确的结构洞察力.
- 现有的展开方法显示实际性能增长,但理论理解有限.
研究的目的:
- 从微分方程 (DE) 的角度探索展开的线性化ADMM (LADMM).
- 为了设计基于DE洞察力的新,更高效的展开深度网络.
- 建立连接展开的ADMM与DE的理论保证.
主要方法:
- 提出了一个展开的欧勒LADMM方案和一个更准确的基于梯形离谱的梯形LADMM方案.
- 使用预测纠正策略开发了形LADMM方案的明确版本.
- 设计了加速的欧勒和梯形LADMM变体,可解释为二级DE,以扩展网络表示能力.
- 实施的方案为 (A-) ELADMM和 (A-) TLADMM与近位运算符以及 (A-) ELADMM-Net和 (A-) TLADMM-Net与卷积神经网络 (CNN).
主要成果:
- 证明了展开的ADMM和第一 (二) 级DE之间具有理论保证的全面联系.
- 与现有方法相比,在广泛的反向问题实验中,拟议的梯形LADMM方案 (A-TLADMM) 在广泛的反向问题实验中表现出卓越的性能.
- 加快计划扩大了展开网络的代表空间,提高了能力.
结论:
- 该研究提供了第一个理论框架,将展开的ADMM与DE联系起来,并提供了对网络结构的见解.
- 新的梯形LADMM方案及其加速变体显著提高了反向问题的性能.
- 这项工作为设计更高效和可解释的深度学习模型为优化任务铺平了道路.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
53
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
53
Deconvolution
159
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
159
Linear Approximation in Frequency Domain
89
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
89
Linear Approximation in Time Domain
81
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
81
Inverse z-Transform by Partial Fraction Expansion
325
The inverse z-transform is a crucial technique for converting a function from its z-domain representation back to the time domain. One effective method for finding the inverse z-transform is the Partial Fraction Method, which involves decomposing a function into simpler fractions with distinct coefficients. These fractions correspond to known z-transform pairs, facilitating the inverse transformation process.
To begin the process, the poles of the function are identified and the function is...
To begin the process, the poles of the function are identified and the function is...
325
Convolution: Math, Graphics, and Discrete Signals
249
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
249


