OTVLD-Net: An Omni-Dimensional Dynamic Convolution-Transformer Network for Lane Detection

With the vigorous development of deep learning technology, lane detection tasks have achieved phased results. However, existing lane detection models do not consider the unique geometric and visual features of lanes when dealing with some challenging scenarios, resulting in many difficulties and limitations. To this end, we propose a lane detection network based on full-dimensional convolutional Transformer (OTVLD-Net) to improve the adaptability of the model under extreme road conditions and better handle complex lane topology. In order to extract richer contextual features, we designed ODVT-Net, which uses full-dimensional dynamic convolution combined with improved feature flip fusion layer and non-local network layer, and aggregates lane symmetry features by utilizing the horizontal symmetry of lanes. A feature weight generation mechanism based on Transformer is designed, and a cross-attention mechanism between feature maps and lane requests is added in the decoding stage to enable the network to aggregate global feature information. At the same time, a vanishing point detection module is introduced, and a joint weighted loss function is designed to be trained in coordination with the lane detection task to improve the generalization ability of the lane detection model. Experimental results on the OpenLane and CurveLanes datasets show that the detection effect of the OTVLD-Net model has reached the current advanced level. In particular, the accuracy on the OpenLane dataset is 6.4% higher than the F1 score of the second-ranked model, and the average performance in different challenging scenarios is also improved by 8.9%. At the same time, when ResNet-18 is used as the template feature extraction network, the model achieves a speed of 103FPS and a computing power of 14.2 GFlops, achieving good performance while ensuring real-time performance.