DFormer++:改善RGBD表示学习,用于语义细分
IEEE transactions on pattern analysis and machine intelligence
|January 27, 2026
概括
DFormer++引入了用于RGB-D语义细分的新型预训和微调框架,通过预训练图像深度对来解决表示不匹配问题. 这种方法增强了对准确感知的3D几何编码.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 由于RGB预训练模型和RGB-D数据之间的不匹配,RGB-D语义细分面临挑战.
- 现有的方法往往无法有效编码深度地图中存在的3D几何关系.
研究的目的:
- 提出DFormer++,一个新的预训练和微调框架,用于学习RGB-D语义细分的可转移表示.
- 为了解决RGB-D语义细分中的常见不匹配问题.
主要方法:
- 开发了DFormer++,这是一个使用ImageNet-1K图像深度对来预训练骨干的框架,可以直接编码RGB-D表示.
- 引入了RGB-D注意力阻塞,具有针对编码RGB和深度信息的新型注意力机制.
主要成果:
- DFormer++有效地避免了RGB预训练的骨干对3D几何学的不匹配编码.
- 量身定制的架构减少了多余的参数,实现了高效和准确的RGB-D感知.
- 在三个流行的RGB-D语义细分基准上实现了最先进的性能.
结论:
- 提出的DFormer++框架成功地学习了强大的RGB-D表示.
- 新的架构和预训练策略显著提高了RGB-D语义细分中的性能和效率.
相关概念视频
State Space Representation
558
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
558
Control Volume and System Representations
1.5K
Two key frameworks are employed to analyze mass, energy, and momentum transfer: the control volume approach and the system approach. These frameworks offer different perspectives, depending on whether the focus is on a specific region in space (control volume approach) or a defined mass of fluid (system approach).
The control volume approach considers a stationary region in space through which fluid flows. This region is bounded by a control surface. For instance, in the case of water...
The control volume approach considers a stationary region in space through which fluid flows. This region is bounded by a control surface. For instance, in the case of water...
1.5K
Graphical Representation of Inequalities
200
The graph of the equation where y equals x squared forms a curve known as a parabola. This curve acts as a boundary in the coordinate plane, dividing it into distinct regions based on the relative position of points.When the equality sign in the equation is replaced with an inequality—such as greater than, less than, greater than or equal to, or less than or equal to—the graphical representation changes from a single curve into a broader shaded area that signifies the set of all...
200
Vector Representation of Complex Numbers
544
Complex numbers, represented in Cartesian coordinates, can also be visualized as vectors. These vectors can be expressed in polar form, emphasizing their magnitude and angle. When a complex number is input into a function, the output is another complex number, highlighting the function's zero point from which the vector representation can originate.
Consider a function defined as the product of the complex factors in the numerator divided by the product of the complex factors in the...
Consider a function defined as the product of the complex factors in the numerator divided by the product of the complex factors in the...
544
Graphical and Analytic Representation of Sinusoids
968
Analyzing two sinusoidal voltages with equal amplitude and period but different phases on an oscilloscope, an instrument used to display and analyze waveforms, involves a three-step process.
The first step is measuring the peak-to-peak value, which is twice the amplitude of the sinusoid. This provides information about the maximum voltage swing of the waveform.
Secondly, the period and angular frequency are determined. The period is the time taken for one complete cycle of the waveform, while...
The first step is measuring the peak-to-peak value, which is twice the amplitude of the sinusoid. This provides information about the maximum voltage swing of the waveform.
Secondly, the period and angular frequency are determined. The period is the time taken for one complete cycle of the waveform, while...
968
Avoidance Learning and Learned Helplessness
2.6K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.6K


