玛丽戈德:基于扩散的图像生成器的经济适应,用于图像分析
概括
玛丽戈德适应大型文本到图像模型用于计算机视觉任务,如深度估计. 这种方法利用基础模型在数据稀缺的场景中进行有效的转移学习,实现最先进的零射击概括.
科学领域:
- 计算机视觉 计算机视觉
- 深度学习 (Deep Learning) 是一种深度学习.
- 生成型模型 生成型模型
背景情况:
- 深度学习的成功依赖于大数据集和预训练模型,特别是在数据稀缺的环境中.
- 传统的预训练使用图像分类和自我监督学习.
- 文本到图像扩散模型提供了一个新的基础模型类别,具有视觉理解.
研究的目的:
- 介绍玛丽戈尔德,一种适应隐性扩散模型用于密度图像分析的方法.
- 能够有效地将学习从大规模的生成模型转移到特定的视觉任务.
主要方法:
- 开发了Marigold,一种条件生成模型家族.
- 引入了微调协议,以从预训练的隐性扩散模型 (例如,稳定扩散) 中提取知识.
- 适应模型的任务,如单眼深度估计,表面正常预测,和内在分解.
主要成果:
- 玛丽戈德需要对预训练模型进行最小的建筑修改.
- 训练是高效的,在几天内使用单个GPU上的小型合成数据集.
- 在密集图像分析任务上实现了最先进的零射击概括性能.
结论:
- 潜在扩散模型可以有效地用于密集的计算机视觉任务.
- 玛丽戈德提供了一种计算效率高,高性能的知识转移方法.
- 这项工作展示了生成模型作为各种视觉应用的强大骨干的潜力.
相关概念视频
01:12Diffusion
176.6K
Diffusion is the passive movement of substances down their concentration gradients—requiring no expenditure of cellular energy. Substances, such as molecules or ions, diffuse from an area of high concentration to an area of low concentration in the cytosol or across membranes. Eventually, the concentration will even out, with the substance moving randomly but causing no net change in concentration. Such a state is called dynamic equilibrium, which is essential for maintaining overall...
176.6K
02:16Light Acquisition
8.0K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
8.0K