Exact Tile-Based Segmentation Inference for Images Larger than GPU Memory.

Area of Science:

• Computer Vision
• Medical Image Analysis
• Deep Learning

Background:

• Fully convolutional neural networks (FCNs) enable semantic segmentation of arbitrarily large images but are limited by GPU memory.
• Processing large images like whole slide microscopy images requires out-of-core methods to overcome memory constraints.

Purpose of the Study:

• To enable exact (tiling-error free) out-of-core semantic segmentation inference for arbitrarily large images using FCNs.
• To overcome GPU memory limitations without introducing numerical errors in the final segmentation results.

Main Methods:

• A tiling strategy using a halo border around each tile, with tile size determined by GPU memory and network receptive field.
• Overlapping input tile halos and precisely joining output tiles at seams to ensure continuity.
• Validation on U-Net and FC-DenseNet architectures, including quantification of tiling-related errors.

Main Results:

• Formulas for calculating optimal tile size and stride were documented.
• The proposed method successfully performed exact semantic segmentation on large images, demonstrated on U-Net and FC-DenseNet.
• Tiling configurations violating constraints were quantified for their error impact.

Conclusions:

• The developed tiling method effectively addresses GPU memory constraints for FCN-based semantic segmentation of large images.
• This approach is suitable for processing high-resolution medical images, such as those from whole slide scanners.
• The study provides a framework for estimating tiling parameters using effective receptive fields of neural network architectures.