雷MVSNet++:学习基于射线的1D隐含场,用于准确的多视图立体声
概括
RayMVSNet通过学习相机射线沿着1D隐性场优化了深度估计,减少了多视图立体 (MVS) 的计算. 这种新的方法在具有挑战性的数据集上取得了最先进的结果,提高了3D重建质量.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 三维重建的3D重建
背景情况:
- 基于学习的多视图立体 (MVS) 方法通常依赖于计算密集的3D卷曲.
- 高的计算和内存需求限制了现有的MVS技术中深度图的分辨率.
- 直接优化成本量对于高分辨率输出来说是资源限制性的.
研究的目的:
- 开发一种更有效,更准确的基于学习的MVS方法.
- 为了减少与传统的MVS方法相关的计算复杂性.
- 在具有挑战性的场景中实现高质量的深度估计和3D重建.
主要方法:
- 提出RayMVSNet,通过学习1D隐含场,优化相机射线的深度.
- 利用变压器功能进行基于射线的顺序深度预测,模仿极线路搜索.
- 包含多任务学习,并利用签名距离函数 (SDF) 的单调性来提高准确性.
- 引入 RayMVSNet++ 带有注意力关门单元,用于增强上下文特征聚合.
主要成果:
- 雷MVSNet在DTU和坦克和寺数据集上取得了最高排名,分别获得0.33毫米和59.48%的F-score.
- 展示了高质量的深度估计和点云重建,用于非纹理,封闭和不同深度场景.
- 雷MVSNet++在ScanNet上实现了最先进的性能,实现了0.058m的AbsRel.
- 在没有纹理的区域和深度差异很大的场景上获得准确的结果.
结论:
- 基于射线的深度优化为MVS中的成本体积优化提供了一个轻量级和有效的替代方案.
- 拟议的RayMVSNet和RayMVSNet++显著提升了基于学习的MVS的最新技术.
- 这些方法对具有挑战性的现实世界条件,包括灯光不良和运动模糊,具有强大性能.
相关概念视频
Depth Perception and Spatial Vision
735
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
735
Inertial Frames of Reference
7.2K
Newton’s first law is usually considered to be a statement about reference frames. It provides a method for identifying a special type of reference frame: the inertial reference frame. In principle, we can make the net force on a body zero. If its velocity relative to a given frame is constant, then that frame is said to be inertial. So, by definition, an inertial reference frame is a reference frame where Newton's first law holds valid. Newton's first law applies to objects with...
7.2K
Sight Distance in a Vertical Curve
80
Sight distance on vertical curves is critical in roadway design. It ensures drivers can see far enough ahead to identify and respond to hazards effectively. This directly impacts safety, driver comfort, and the overall efficiency of the transportation network.Vertical curves are classified into crest and sag curves based on their geometry. For crest curves, sight distance is determined by the line of sight between a driver's eye and a small object on the road's surface. Design parameters for...
80
Uniform Depth Channel Flow: Problem Solving
90
To calculate the flow rate for a trapezoidal channel, first, identify the bottom width, side slope, and flow depth of the channel. The cross-sectional area (A) corresponding to the depth of flow (y), channel bottom width (B), and side slope (θ) is determined by:Next, calculate the wetted perimeter, which includes the bottom width and the sloped side lengths in contact with the water. Using the values of the cross-sectional area and the wetted perimeter, determine the hydraulic radius by...
90
Scalar and Vector Triple Products
2.5K
Two vectors can be multiplied using a scalar product or a vector product. The resultant of a scalar product is scalar, while with vector products, the resultant is a vector. These rules of the scalar or vector product between two vectors can be applied to multiple vectors to obtain meaningful combinations. The scalar triple product is the dot product of a vector with the cross product of two vectors.
The scalar triple product is the dot product of a vector with the cross product of two vectors....
The scalar triple product is the dot product of a vector with the cross product of two vectors....
2.5K
Spherical Coordinates
10.4K
Spherical coordinate systems are preferred over Cartesian, polar, or cylindrical coordinates for systems with spherical symmetry. For example, to describe the surface of a sphere, Cartesian coordinates require all three coordinates. On the other hand, the spherical coordinate system requires only one parameter: the sphere's radius. As a result, the complicated mathematical calculations become simple. Spherical coordinates are used in science and engineering applications like electric and...
10.4K


