Systematic Evaluation of Image Tiling Adverse Effects on Deep Learning Semantic Segmentation.

Area of Science:

• Computer Vision
• Deep Learning
• Medical Imaging

Background:

• Convolutional neural network (CNN) models excel at image tasks but face hardware memory limitations for large images.
• U-Net and similar models are often trained on down-sampled images or tiled inputs due to memory constraints.
• Tiling large images into smaller segments for CNN inference can introduce performance-degrading variations.

Purpose of the Study:

• To investigate the impact of image tiling on CNN performance for medical and satellite imaging.
• To quantify inference variations caused by tiling and the translational invariance of CNNs.
• To compare the effectiveness of 2D and 3D semantic segmentation models.

Main Methods:

• Quantified inference variations on medical (BraTS) and satellite datasets using tiled input.
• Trained and evaluated 2D U-Net models on whole images versus tiled images.
• Compared 2D and 3D CNN models for semantic segmentation accuracy and consistency.

Main Results:

• Tiling CNN inputs introduces small but significant variations detrimental to model performance.
• Training 2D U-Net models on whole images substantially improves segmentation performance.
• 3D CNN models provide more accurate and consistent predictions by leveraging wider image context.

Conclusions:

• Image tiling, while necessary for hardware memory limits, can cause unpredictable CNN output errors.
• Maximizing input tile size or training on whole images is crucial for mitigating these errors.
• 3D CNNs offer superior performance by incorporating volumetric context, addressing limitations of 2D tiling approaches.