Maritime urban tracking dataset in harbor environment

Maritime target tracking datasets are crucial for developing and benchmarking algorithms that support safe navigation of intelligent marine vessels in congested environments. Unlike the automotive domain, where benchmarks like KITTI have driven progress, maritime datasets with ground truth remain limited. This paper introduces the Maritime Urban Tracking (MUT) dataset for perception and tracking in urban waters for autonomous surface vessels. Data were collected using an autonomous ferry prototype for stereo matching, optical flow, Simultaneous Localization and Mapping (SLAM), 2D/3D object detection, water segmentation, and tracking. The ego-vessel is equipped with short- and wide-baseline stereo cameras, Light Detection And Ranging (LiDAR), Realt-Time Kinematic (RTK) Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), and a polarized stereo rig. Some targets feature dual GNSS receivers with Post-Processed Kinematics (PPK) for accurate world-frame reference tracks. The dataset includes 19 tracking scenarios, 8 calibration sequences, 1 mapping scenario, and 3 docking scenarios, mostly 1 minute long, recorded at 30 fps (cameras) and 10 Hz (Lidar). Public release aims to lower entry barriers and foster innovation.