UAV-Based Image and LiDAR Fusion for Pavement Crack Segmentation

Pavement surface maintenance is pivotal for road safety. There exist a number of manual, time-consuming methods to examine pavement conditions and spot distresses. More recently, alternative pavement monitoring methods have been developed, which take advantage of unmanned aerial systems (UASs). However, existing UAS-based approaches make use of either image or LiDAR data, which do not allow for exploring the complementary characteristics of the two systems. This study explores the feasibility of fusing UAS-based imaging and low-cost LiDAR data to enhance pavement crack segmentation using a deep convolutional neural network (DCNN) model. Three datasets are collected using two different UASs at varying flight heights, and two types of pavement distress are investigated, namely cracks and sealed cracks. Four different imaging/LiDAR fusing combinations are created, namely RGB, RGB + intensity, RGB + elevation, and RGB + intensity + elevation. A modified U-net with residual blocks inspired by ResNet was adopted for enhanced pavement crack segmentation. Comparative analyses were conducted against state-of-the-art networks, namely U-net and FPHBN networks, demonstrating the superiority of the developed DCNN in terms of accuracy and generalizability. Using the RGB case of the first dataset, the obtained precision, recall, and F-measure are 77.48%, 87.66%, and 82.26%, respectively. The fusion of the geometric information from the elevation layer with RGB images led to a 2% increase in recall. Fusing the intensity layer with the RGB images yielded a reduction of approximately 2%, 8%, and 5% in the precision, recall, and F-measure. This is attributed to the low spatial resolution and high point cloud noise of the used LiDAR sensor. The second dataset crack samples obtained largely similar results to those of the first dataset. In the third dataset, capturing higher-resolution LiDAR data at a lower altitude led to improved recall, indicating finer crack detail detection. This fusion, however, led to a decrease in precision due to point cloud noise, which caused misclassifications. In contrast, for the sealed crack, the addition of LiDAR data improved the sealed crack segmentation by about 4% and 7% in the second and third datasets, respectively, compared to the RGB cases.