LMFFNet: A Well-Balanced Lightweight Network for Fast and Accurate Semantic Segmentation

Area of Science:

• Computer Vision
• Deep Learning
• Robotics

Background:

• Real-time semantic segmentation is crucial for autonomous driving and robotics.
• Existing accurate networks are computationally intensive, while lightweight networks compromise accuracy.
• A balance between parameters and accuracy is essential for real-time semantic segmentation.

Purpose of the Study:

• To propose a novel lightweight multiscale-feature-fusion network (LMFFNet) for efficient real-time semantic segmentation.
• To achieve a balance between model complexity (parameters) and segmentation accuracy.
• To demonstrate the effectiveness of LMFFNet across various datasets and resolutions.

Main Methods:

• The proposed LMFFNet utilizes a split-extract-merge bottleneck (SEM-B) block for efficient feature extraction.
• Feature fusion modules (FFMs) are employed to integrate multiscale semantic features, enhancing accuracy.
• A multiscale attention decoder (MAD) is incorporated to recover image details via an attention mechanism.

Main Results:

• LMFFNet-3-8 achieved 75.1% mean intersection over union (mIoU) with only 1.4 million parameters.
• The model demonstrated a high inference speed of 118.9 frames/s on an RTX 3090 GPU without pretraining.
• Experiments on CamVid, KITTI, and WildDash2 datasets confirmed the model's performance across different resolutions.

Conclusions:

• LMFFNet offers a significant tradeoff between segmentation accuracy and inference speed for real-time applications.
• The proposed architecture effectively addresses the limitations of existing lightweight and complex semantic segmentation models.
• The study provides a viable solution for efficient semantic segmentation in resource-constrained environments like autonomous systems.