SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Existing neural networks often have high computational costs and memory consumption.
• Dynamically adjusting image processing resolution can potentially improve efficiency.
• Feature discontinuities at block borders are a challenge in block-based image processing.

Purpose of the Study:

• To introduce SegBlocks, a novel method for reducing the computational cost of neural networks.
• To improve the accuracy-complexity trade-off in image processing tasks.
• To address feature discontinuities in block-based processing.

Main Methods:

• SegBlocks splits images into blocks and downsamples low-complexity regions.
• A reinforcement learning-trained policy network identifies complex regions.
• CUDA-implemented modules and a novel BlockPad module handle block processing and prevent discontinuities.

Main Results:

• Experiments on Cityscapes, Camvid, and Mapillary Vistas datasets demonstrate improved accuracy-complexity trade-offs.
• SegBlocks reduced floating-point operations by 60% and increased inference speed by 50% for SwiftNet-RN18.
• A minimal 0.3% decrease in mIoU accuracy was observed on Cityscapes.

Conclusions:

• SegBlocks offers a significant reduction in computational cost and memory consumption for neural networks.
• The BlockPad module effectively prevents feature discontinuities at block borders.
• Dynamically processing images with SegBlocks provides a superior accuracy-complexity trade-off compared to static methods.