Umono3r: transformer guided by a transmission map for underwater monocular 3D reconstruction

Underwater 3D reconstruction technology is widely used in seabed exploration, AUV path planning, and other fields. However, the complex scattering and absorption effects of the water body often lead to color distortion and blurring in underwater images. The robustness of degraded underwater image sequences based on traditional multi-view reconstruction methods is not good. In addition, the scarcity of underwater 3D datasets also poses a challenge to the research. Based on the above problems, we propose the first end-to-end monocular underwater 3D reconstruction network Umono3r, which is based on the vision transformer codec architecture and configured with a transmission-guided cross-attention (TGCA) module to fuse underwater transmittance information to enhance the understanding of the underwater scene, and then trained on our constructed high-quality underwater 3D dataset Flsea3D. Experimental results demonstrate the effectiveness of our proposed Umono3r for reconstructing 3D scenes, which outperforms the performance of existing unconstrained underwater monocular reconstruction algorithms qualitatively and quantitatively.