Fusion of Multimodal Imaging and 3D Digitization Using Photogrammetry

Area of Science:

• Optics and Photonics
• Computer Vision
• 3D Reconstruction

Background:

• Multimodal sensors capture diverse scene characteristics, but integrating their data with high-resolution 3D models is challenging.
• Low resolution and image quality of multimodal cameras (hyperspectral, thermal) limit their direct use with detailed 3D data.

Purpose of the Study:

• To develop and demonstrate a new method for superimposing multimodal image data onto 3D models generated via multi-view photogrammetry.
• To overcome the resolution limitations of multimodal cameras in 3D digitization applications.

Main Methods:

• A novel superimposition technique was developed to fuse low-resolution multimodal imagery with high-resolution 3D models.
• A laboratory setup integrated a high-resolution photo camera, a thermal camera, and a 12-channel multispectral camera on a calibrated rig.
• Multi-view photogrammetry was used to create a detailed 3D model, onto which multimodal data was superimposed.

Main Results:

• The data fusion method achieved an accuracy better than one pixel for multimodal superimposition.
• Successful demonstration of multimodal 3D digitization using integrated camera systems.

Conclusions:

• The proposed method effectively integrates multimodal sensor data with high-resolution 3D models, enhancing information gain.
• This approach offers a pathway for advanced 3D digitization with improved material and condition analysis.