A Dataset for Fish Segmentation and Tracking in Underwater Videos

The automatic monitoring of fish in underwater imagery plays a key role in marine ecology, fisheries management, and environmental monitoring, yet progress is limited by the lack of large, high-quality fish-focused datasets. We present a new dataset of underwater videos of fish in natural habitats, annotated for pixel-level segmentation and multi-object tracking. The data was collected in the Balearic Sea, the western Mediterranean region surrounding the island of Mallorca (Spain), across diverse marine environments to capture variations in species, lighting, turbidity, and background complexity. Each video frame has been carefully annotated to ensure spatial and temporal consistency, yielding a challenging and comprehensive resource for developing and benchmarking underwater vision algorithms. To illustrate the dataset's utility, we provide baseline tracking results obtained with Deep OC-SORT, which highlight both the dataset's challenging nature and its potential for future method evaluation. In addition, we release an open-source, browser-based annotation tool integrating the Segment Anything Model (SAM2) and CUTIE for efficient semi-automatic segmentation and tracking. This tool facilitates high-quality annotations without specialized hardware, improving accessibility and reproducibility within the marine imaging community.