Color Image Generation from LiDAR Reflection Data by Using Selected Connection UNET.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Generating realistic color images from sparse Light Detection and Ranging (LiDAR) data is challenging.
• Existing methods like asymmetric encoder-decoder fully convolutional networks (ED-FCN) have limitations with extremely sparse data.

Purpose of the Study:

• To propose and evaluate a modified Selected Connection UNET (SC-UNET) for generating camera-like color images from LiDAR reflection images.
• To investigate various encoder-decoder connection strategies within the SC-UNET architecture.

Main Methods:

• Implementation of a SC-UNET architecture tailored for heterogeneous image generation.
• Analysis of network connection methodologies considering encoder sparseness and inter-level similarity.
• Comparative evaluation against conventional asymmetric ED-FCN using peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

Main Results:

• The proposed SC-UNET with connections at the two lowest levels achieved significant improvements.
• An increase of 3.87 dB in PSNR and 0.17 in SSIM was observed compared to the asymmetric ED-FCN.
• The method effectively addresses the challenge of generating high-quality images from extremely sparse LiDAR data (only 5.28% valid values).

Conclusions:

• The developed SC-UNET methodology offers a powerful solution for generating data from heterogeneous sources, particularly from sparse LiDAR.
• The network connection strategy is crucial for optimizing performance with sparse input data.
• This work advances the capability of cross-modal image generation.