MResTNet: A Multi-Resolution Transformer Framework with CNN Extensions for Semantic Segmentation.

Area of Science:

• Computer Vision
• Deep Learning

Background:

• Semantic segmentation is crucial for object identification in visual scenes.
• Transformer networks outperform traditional Convolutional Neural Networks (CNNs) in segmentation.
• Current high-performance models are computationally intensive, hindering real-time use.

Purpose of the Study:

• To develop a semantic segmentation model balancing high performance with low computational complexity.
• To enable real-time applications like autonomous driving.

Main Methods:

• Proposed a model with a visual transformer encoder and a parallel twin decoder.
• The twin decoder comprises a visual transformer decoder and a CNN decoder.
• Integrated decoders using trainable CNN blocks: a 'fuser' and a 'scaler'.

Main Results:

• Achieved state-of-the-art performance on Cityscapes and ADE20K datasets.
• Demonstrated a low-complexity network architecture.
• The model is suitable for real-time applications.

Conclusions:

• The proposed parallel twin decoder architecture effectively enhances semantic segmentation.
• Achieved a balance between performance and computational efficiency.
• The model is viable for real-time computer vision tasks.