Nessys: A new set of tools for the automated detection of nuclei within intact tissues and dense 3D cultures

Methods for measuring the properties of individual cells within their native 3D environment will enable a deeper understanding of embryonic development, tissue regeneration, and tumorigenesis. However, current methods for segmenting nuclei in 3D tissues are not designed for situations in which nuclei are densely packed, nonspherical, or heterogeneous in shape, size, or texture, all of which are true of many embryonic and adult tissue types as well as in many cases for cells differentiating in culture. Here, we overcome this bottleneck by devising a novel method based on labelling the nuclear envelope (NE) and automatically distinguishing individual nuclei using a tree-structured ridge-tracing method followed by shape ranking according to a trained classifier. The method is fast and makes it possible to process images that are larger than the computer's memory. We consistently obtain accurate segmentation rates of >90%, even for challenging images such as mid-gestation embryos or 3D cultures. We provide a 3D editor and inspector for the manual curation of the segmentation results as well as a program to assess the accuracy of the segmentation. We have also generated a live reporter of the NE that can be used to track live cells in 3 dimensions over time. We use this to monitor the history of cell interactions and occurrences of neighbour exchange within cultures of pluripotent cells during differentiation. We provide these tools in an open-access user-friendly format.