Deep learning-based end-to-end 3D depth recovery from a single-frame fringe pattern with the MSUNet++ network

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Single-frame fringe pattern analysis is crucial for 3D depth reconstruction.
• Existing methods face challenges in achieving high precision and preserving fine geometric details due to limited information.

Purpose of the Study:

• To propose an end-to-end deep learning (DL)-based method for accurate 3D depth recovery from single fringe patterns.
• To enhance the preservation of geometry details and high-frequency signals in reconstructed 3D shapes.

Main Methods:

• A multi-scale feature fusion convolutional neural network (CNN), MSUNet++, was developed.
• Discrete Wavelet Transform (DWT) was incorporated for preprocessing fringe patterns to extract high-frequency signals.
• A novel loss function combining structural similarity and edge perception was designed.

Main Results:

• The proposed MSUNet++ method demonstrated significantly enhanced high-frequency geometry details.
• The method effectively maintained the overall geometric shape of objects.
• Ablation studies and generalization experiments confirmed the method's accuracy, detail preservation, and robustness.

Conclusions:

• The developed MSUNet++ method offers a robust and accurate solution for 3D depth reconstruction from single fringe patterns.
• The integration of DWT and a specialized loss function improves the recovery of fine geometric features.
• The method shows superior performance compared to existing approaches in accuracy and detail preservation.