Dual-Task ConvLSTM-UNet for Instance Segmentation of Weakly Annotated Microscopy Videos

Area of Science:

• Biomedical Image Analysis
• Computational Biology
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel at segmenting biomedical images, including living cell microscopy sequences, due to their ability to capture structural data.
• Standard CNNs often overlook temporal information in time-lapse sequences, which is vital for distinguishing touching or overlapping cells.
• Exploiting cell dynamics is crucial for advancing cell segmentation accuracy in dynamic biological processes.

Purpose of the Study:

• To develop a novel CNN architecture that effectively utilizes temporal information from time-lapse microscopy sequences for improved cell segmentation.
• To create a dual-task network capable of training with weakly annotated data, such as cell centers (markers), when complete cell outlines are unavailable.
• To adapt the segmentation method for 3D microscopy data without significant computational overhead.

Main Methods:

• Proposed a novel Recurrent Neural Network (RNN) architecture by integrating Convolutional Long Short-Term Memory (ConvLSTM) with the U-Net (ConvLSTM-U-Net).
• Developed a dual-task network design for training with approximate cell center annotations (markers).
• Employed the fast marching method for segmenting clustered cells and adapted the approach for 3D microscopy data.

Main Results:

• The ConvLSTM-U-Net ranked among the top three methods on six datasets in the Cell Segmentation Benchmark.
• Achieved state-of-the-art cell detection results on a publicly available, weakly annotated dataset using the built-in marker estimator.
• Demonstrated effective segmentation of complex cellular structures, low-contrast images, and unclear boundaries by leveraging spatio-temporal features.

Conclusions:

• The proposed ConvLSTM-U-Net effectively exploits cell dynamics and temporal information for superior cell segmentation in microscopy.
• The dual-task architecture and marker-based training enable robust performance with weakly annotated data.
• The method shows promise for both 2D and 3D time-lapse microscopy, offering a significant advancement in automated cell analysis.