导向图像反向问题的多模态卷积参数化网络
Mikolaj Czerkawski1, Priti Upadhyay1, Christopher Davison1
1Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UK.
Journal of imaging
|March 27, 2024
概括
深度内部学习使用单个样本解决图像反向任务. 一个新的多模组卷积参数化网络 (MCPN) 提高了多模组数据的性能,用于诸如绘制卫星图像和超分辨率等任务.
科学领域:
- 计算机视觉 计算机视觉
- 深度学习 (Deep Learning) 是一种深度学习.
- 图像处理 图像处理
背景情况:
- 深度内部学习利用在样本本身上训练的深度神经网络,而不是外部数据集,用于像 inpainting 和超分辨率等图像反向任务.
- 现有的方法,如Deep Image Prior,在处理多模式图像数据时是有限的.
- 多模式表示在诸如卫星图像处理等领域至关重要.
研究的目的:
- 提出一个新的多模态卷积参数化网络 (MCPN) 架构.
- 为了增强多模式图像反向问题的深度内部学习.
- 为了证明MCPN在单模式方法上的有效性.
主要方法:
- 开发了MCPN,一个卷积神经网络,集成了一个共享的核心网络与特定模式的头部.
- MCPN 在多个数据模式中近似共享信息.
- 应用于MCPN引导图像反向问题,包括inpainting和超分辨率.
主要成果:
- MCPN显著优于单模卷积参数化网络.
- 拟议的方法在多模式引导图像反向任务上表现出卓越的性能.
- 通过使用多式联络数据,在绘画和超分辨率方面取得了改进的结果.
结论:
- 对于图像反向任务,MCPN有效地处理多模式数据.
- 拟议的网络架构推进了用于复杂图像处理的深度内部学习.
- MCPN为卫星图像处理和其他多模式应用提供了显著的改进.
相关概念视频
01:24Convolution: Math, Graphics, and Discrete Signals
250
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
250
01:20Deconvolution
159
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
159
01:20Convolution Properties I
147
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
147
01:17Convolution Properties II
198
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
198