Related Experiment Video
Updated: Jul 31, 2026
04:23
A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
Published on: April 21, 2023
1.8K
Multi-Task Foreground-Aware Network with Depth Completion for Enhanced RGB-D Fusion Object Detection Based on
Jiasheng Pan1, Songyi Zhong2,3, Tao Yue2
1School of Computer Engineering and Science, Shanghai University, No. 99 Shangda Road, Shanghai 200444, China.
Sensors (Basel, Switzerland)
|April 13, 2024
Summary
This study introduces a Transformer-based neural network for enhanced autonomous driving perception. The model improves object detection by fusing LiDAR and camera data, especially for small and distant targets.
Area of Science:
- Computer Vision
- Robotics
- Artificial Intelligence
Background:
- Autonomous driving systems rely on sensor fusion, primarily LiDAR and cameras, for target recognition.
- Traditional methods struggle with sparse LiDAR data, hindering detection of small or distant objects.
Purpose of the Study:
- To develop a multi-task parallel neural network for simultaneous depth completion and object detection.
- To improve the fusion of LiDAR and camera data for more robust autonomous driving perception.
Main Methods:
- A Transformer-based multi-task parallel neural network was designed.
- Redesigned loss functions to minimize environmental noise in depth completion.
- Introduced a novel fusion module to enhance foreground-background perception.
Main Results:
- The network effectively completes LiDAR point clouds using RGB pixel correlations, addressing feature mismatches.
- Achieved significant performance improvements: 4.78% for cars, 8.93% for pedestrians, and 15.54% for cyclists.
- Demonstrated a processing speed of 38 frames per second (fps), indicating efficiency.
Conclusions:
- The proposed method enhances object detection, particularly for challenging targets, by effectively fusing multi-sensor data.
- The network's ability to perform depth completion and object detection simultaneously offers a feasible and efficient solution for autonomous driving.
