Multi-sensor fusion and segmentation for autonomous vehicle multi-object tracking using deep Q networks

Autonomous vehicles, often known as self-driving cars, have emerged as a disruptive technology with the promise of safer, more efficient, and convenient transportation. The existing works provide achievable results but lack effective solutions, as accumulation on roads can obscure lane markings and traffic signs, making it difficult for the self-driving car to navigate safely. Heavy rain, snow, fog, or dust storms can severely limit the car's sensors' ability to detect obstacles, pedestrians, and other vehicles, which pose potential safety risks. So, we have presented a multi-sensor fusion and segmentation for multi-object tracking using DQN in self-driving cars. Our proposed scheme incorporates the handling of pipelines for camera and LiDAR data and the development of an autonomous solution for object detection by handling sensor images. An Improved Adaptive Extended Kalman Filter (IAEKF) was used for noise reduction. The Contrast enhancement was done using a Normalised Gamma Transformation based CLAHE (NGT-CLAHE), and the adaptive thresholding was implemented using an Improved Adaptive Weighted Mean Filter (IAWMF) which was used for preprocessing. The multi-segmentation based on orientation employs various segmentation techniques and degrees. The dense net-based multi-image fusion gives more efficiency and a high memory in terms of fast processing time. The Energy Valley Optimizer (EVO) approach is used to select grid map-based paths and lanes. This strategy solves complicated tasks in a simple manner, which leads to ease of flexibility, resilience, and scalability. In addition, the YOLO V7 model is used for detection and categorization. The proposed work is evaluated using metrics such as velocity, accuracy rate, success rate, success ratio, mean squared error, loss rate, and accumulated reward.