NucVerse3D: generalizable 3D nuclear instance segmentation across heterogeneous microscopy modalities.

Area of Science:

• Biomedical imaging
• Computational biology
• Machine learning

Background:

• Accurate 3D nuclear instance segmentation is crucial for quantitative phenotyping in volumetric microscopy.
• Existing methods face challenges with dense tissues, irregular nuclei, and varied imaging modalities.

Purpose of the Study:

• To develop a generalized deep learning framework, NucVerse3D, for robust 3D nuclear instance segmentation across multiple imaging modalities.
• To enable quantitative 3D nuclear phenotyping for biomedical applications, including cancer research.

Main Methods:

• NucVerse3D utilizes a residual attention 3D U-Net architecture with a reversible gradient-field representation.
• The framework employs modality-agnostic preprocessing and isotropic scale normalization for broad applicability.
• Trained end-to-end in 3D on diverse datasets, including newly released ground-truth data.

Main Results:

• NucVerse3D demonstrated high precision and competitive recall across seven volumetric datasets, achieving strong F1-scores.
• A single generalized model performed comparably to dataset-specific models.
• Application to hepatocellular carcinoma models revealed disrupted nuclear DNA-volume scaling, quantified by a novel Nuclear Decoupling Score (NDS).

Conclusions:

• NucVerse3D provides a robust and generalizable solution for 3D nuclear instance segmentation.
• The framework facilitates quantitative nuclear phenotyping, offering new biomarkers like NDS for pathological tissue analysis.
• NucVerse3D advances the analysis of volumetric biological data across various imaging techniques.