LLM-enhanced Neuron Segmentation and Reconstruction in Complex Mouse Brain Images

Area of Science:

• Neuroscience
• Computational Biology
• Medical Imaging

Background:

• Accurate neuron segmentation is crucial for understanding brain structure and function.
• Existing deep learning methods often fail to fully utilize prior information like neuronal morphology and imaging mechanisms.
• Complex mouse brain images present challenges for precise neuron segmentation and reconstruction.

Purpose of the Study:

• To develop an LLM-integrated framework, NUNet-LLM, for enhanced neuron segmentation and reconstruction.
• To improve the exploitation of prior information, including neuronal morphology and imaging mechanisms, in deep learning models.
• To provide a robust solution for analyzing complex neuronal structures in mouse brain images.

Main Methods:

• Proposed NUNet-LLM, a framework combining a large language model (LLM)-based text path and a 3D UNet-based image path.
• The text path pre-computes static textual features using two pre-trained LLMs for dataset and task descriptions.
• The image path employs a 3D UNet with wavelet transform and attention mechanisms, fusing image and textual features for segmentation.
• Developed a novel topology structure loss combining cross-entropy, structure loss, and edge-aware loss.
• Constructed mouse brain neuronal cube dataset (mNeuCuDa) and a synthetic dataset (sNeuCuDa).
• Utilized an automatic algorithm and G-Cut for neuron reconstruction and decoupling.

Main Results:

• NUNet-LLM effectively segments neurons in complex mouse brain images.
• The framework demonstrates superior performance in neuron reconstruction compared to existing methods.
• Experiments on mouse brain images and the BigNeuron dataset validate the model's effectiveness.
• The fusion of multi-modal features significantly guides the deep model to focus on slender nerve fibers.

Conclusions:

• NUNet-LLM represents a significant advancement in neuron segmentation and reconstruction by integrating LLMs.
• The proposed framework enhances the analysis of neuronal morphology and brain structure.
• NUNet-LLM offers a powerful tool for neuroscience research, improving the accuracy and efficiency of neuron reconstruction studies.