Fringe projection profilometry by conducting deep learning from its digital twin

Area of Science:

• Optics and Photonics
• Computer Vision
• Metrology

Background:

• Three-dimensional (3D) fringe projection profilometry (FPP) is crucial for high-accuracy, high-speed measurements.
• Deep learning models enhance fringe analysis accuracy but require extensive training datasets.
• Acquiring sufficient real-world training data for FPP is costly and labor-intensive.

Purpose of the Study:

• To develop a cost-effective and efficient method for generating training data for deep learning-based FPP.
• To propose a virtual scanning approach using computer graphics and digital twins of FPP systems.
• To enable direct 3D geometry extraction from single-shot fringe images using virtually trained models.

Main Methods:

• Construction of a digital twin for a fringe projection profilometry system.
• Utilizing computer graphics for virtual scanning to generate synthetic fringe images and 3D scenes.
• Training deep learning models on virtually generated data for 3D reconstruction.
• Direct extraction of 3D geometry from single-shot fringe images.

Main Results:

• Successful demonstration of a virtually trained deep learning model on real-world experiments.
• Automatic generation of 7,200 fringe images and 800 corresponding 3D scenes within 1.5 hours.
• Significant reduction in cost and labor associated with data acquisition for FPP.

Conclusions:

• Virtual scanning using digital twins offers a viable solution for generating large-scale training datasets for FPP.
• The proposed method effectively trains deep learning models, enabling accurate 3D reconstruction from single-shot images.
• This approach significantly enhances the efficiency and accessibility of deep learning applications in 3D profilometry.