Related Experiment Videos
RPT-Fusion: A Time-Lag-Aware Quality-Adaptive Radar-Camera Fusion Framework for Water-Surface Object Detection
Yabin Xu1, Sujie Zhan1, Junnan Yang1
1School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, China.
Abstract:
Water-surface object detection remains challenging under strong reflections, wave disturbances, adverse weather, low illumination, and distant or small targets, which can severely degrade or obscure discriminative visual cues. Although radar-camera fusion provides complementary geometric and motion information, sparse and uncertain radar observations may introduce unreliable cues, while camera-radar temporal asynchrony can further reduce cross-modal spatial consistency. To address these issues, this paper proposes Radar Prior and Time-Lag-Aware Quality-Adaptive Fusion (RPT-Fusion), a radar-camera fusion framework that preserves the camera as the primary semantic modality while treating radar as a reliability-controlled auxiliary source. Sparse 4D radar measurements are transformed into probabilistic occupancy, density, velocity, and reliability priors, with direction-dependent spatial uncertainty represented through anisotropic radar-prior modeling. Local, global, and temporal quality estimates are then jointly used to regulate radar contributions during multi-scale feature fusion. In particular, the measured camera-radar time lag is explicitly incorporated into both radar-prior construction and feature-level reliability control. Experiments on WaterScenes show that RPT-Fusion achieves 92.31% mAP50 at an Intersection-over-Union (IoU) threshold of 0.50 and 68.23% mAP50-95 averaged over IoU thresholds from 0.50 to 0.95, outperforming WS-DETR by 0.82 and 3.69 percentage points, respectively. Ablation experiments verify the contributions of radar-prior modeling, quality-adaptive fusion, and temporal quality, while repeated-training experiments show low performance variability. Time-lag analysis reveals condition-dependent benefits, with the largest observed improvement of 1.15 percentage points occurring when the camera-radar offset reaches or exceeds 20 ms. Radar-frame-dropping experiments further show that mAP50 decreases from 92.31% to 91.54% when radar observations are progressively removed, indicating that the camera-centric pathway retains a substantial detection capability under incomplete radar observations. These results demonstrate the effectiveness of reliability-controlled radar assistance for robust radar-camera object detection in complex water-surface environments.
Related Concept Videos
Uniform Depth Channel Flow: Problem Solving
Rapidly Varying Flow