A Dataset for Fish Segmentation and Tracking in Underwater Videos

Area of Science:

• Marine Biology
• Computer Vision
• Ecological Monitoring

Background:

• Automatic fish monitoring is crucial for marine ecology, fisheries management, and environmental oversight.
• Current progress is hampered by a scarcity of extensive, high-quality fish-specific datasets for underwater imagery.
• Developing robust algorithms requires diverse data reflecting natural habitat variations.

Purpose of the Study:

• Introduce a novel dataset of underwater fish videos with pixel-level segmentation and multi-object tracking annotations.
• Provide a challenging and comprehensive resource for the advancement and evaluation of underwater vision algorithms.
• Facilitate efficient and reproducible annotation within the marine imaging community.

Main Methods:

• Collected underwater videos in the Balearic Sea, encompassing diverse marine environments.
• Annotated video frames for pixel-level segmentation and multi-object tracking with spatial and temporal consistency.
• Developed an open-source, browser-based annotation tool integrating Segment Anything Model (SAM2) and CUTIE for semi-automatic annotation.

Main Results:

• Established a new, high-quality dataset of annotated underwater fish videos.
• Demonstrated the dataset's utility and challenging nature through baseline tracking with Deep OC-SORT.
• Released an accessible annotation tool to streamline the creation of marine imaging datasets.

Conclusions:

• The new dataset significantly advances the development and benchmarking of underwater vision algorithms.
• The provided annotation tool enhances accessibility and reproducibility for marine imaging research.
• This resource is expected to accelerate progress in automated fish monitoring for ecological and management applications.