NRTR: Neuron Reconstruction With Transformer From 3D Optical Microscopy Images

Area of Science:

• Neuroscience
• Computer Vision
• Machine Learning

Background:

• Neuron reconstruction from optical microscopy (OM) is fundamental to neuroscience.
• Manual and semi-automatic methods are inefficient; existing deep learning approaches often require complex rule-based components.
• A simpler, end-to-end deep learning method for neuron reconstruction is needed.

Purpose of the Study:

• To develop a novel, end-to-end deep learning model for neuron reconstruction.
• To simplify the neuron reconstruction framework by eliminating complex rule-based components.
• To treat neuron reconstruction as a direct set-prediction problem.

Main Methods:

• Propose the Neuron Reconstruction Transformer (NRTR), an image-to-set deep learning model.
• The NRTR pipeline includes a CNN backbone, Transformer encoder-decoder, and a connectivity construction module.
• NRTR generates a point set representing neuron morphology, with relationships established via connectivity construction, outputting standard SWC files.

Main Results:

• NRTR achieves excellent neuron reconstruction results on the BigNeuron and VISoR-40 datasets.
• The model outperforms competitive baselines in comprehensive benchmarks.
• Demonstrates the effectiveness of the set-prediction approach for end-to-end neuron reconstruction.

Conclusions:

• NRTR offers an effective and simplified end-to-end solution for neuron reconstruction.
• Viewing neuron reconstruction as a set-prediction problem facilitates easier model training.
• The NRTR model advances automated neuron tracing in neuroscience research.