Dual-stage hybrid network for single-shot fringe projection profilometry based on a phase-height model

Single-shot fringe projection profilometry (FPP) is widely used in the field of dynamic optical 3D reconstruction because of its high accuracy and efficiency. However, the traditional single-shot FPP methods are not satisfactory in reconstructing complex scenes with noise and discontinuous objects. Therefore, this paper proposes a Deformable Convolution-Based HINet with Attention Connection (DCAHINet), which is a dual-stage hybrid network with a deformation extraction stage and depth mapping stage. Specifically, the deformable convolution module and attention gate are introduced into DCAHINet respectively to enhance the ability of feature extraction and fusion. In addition, to solve the long-standing problem of the insufficient generalization ability of deep learning-based single-shot FPP methods on different hardware devices, DCAHINet outputs phase difference, which can be converted into 3D shapes by simple multiplication operations, rather than directly outputting 3D shapes. To the best of the author's knowledge, DCAHINet is the first network that can be applied to different hardware devices. Experiments on virtual and real datasets show that the proposed method is superior to other deep learning or traditional methods and can be used in practical application scenarios.