Culling for Extreme-Scale Segmentation Volumes: A Hybrid Deterministic and Probabilistic Approach

Area of Science:

• Neuroscience
• Computer Science
• Data Science

Background:

• Large-scale microscopy generates terabyte-sized datasets, including complex segmentation labels.
• Efficient in-memory representation and random access of integer label data are critical challenges.
• Rapid identification of segment locations (culling) is essential for rendering and spatial queries.

Purpose of the Study:

• To develop a novel approach for efficient culling of millions of labeled segments in large 3D image volumes.
• To reduce storage size and improve query times for segmentation label data.

Main Methods:

• Introduced a hybrid data structure: the label list tree.
• Combined deterministic and probabilistic data representations adaptively within the tree.
• Implemented a query-adaptive culling strategy for efficient data traversal and pruning.

Main Results:

• The label list tree significantly reduces storage size for segmentation labels.
• Achieved faster query times compared to conventional culling approaches.
• Demonstrated efficiency on large connectomics datasets, including a brain scan with over 13 million segments.

Conclusions:

• The proposed hybrid approach offers a scalable solution for managing and querying large-scale segmentation label data.
• This method enables faster analysis and visualization of complex biological datasets.
• The label list tree is a promising data structure for big data challenges in scientific imaging.