CoT: Contourlet Transformer for Hierarchical Semantic Segmentation.

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Transformer-CNN hybrid networks balance deep and shallow image features for hierarchical semantic segmentation.
• Existing methods face challenges in simultaneously achieving comprehensive semantic understanding and meticulous detail extraction.

Purpose of the Study:

• Propose a novel Transformer-CNN hybrid hierarchical network, the Contourlet Transformer (CoT).
• Address the contradiction between semantic understanding and detail extraction in hierarchical semantic segmentation.

Main Methods:

• Utilize Contourlet Transform (CT) to distill high-frequency directional components, creating localized features for CNNs.
• Employ a Deep Detail Representation (DDR) structure with a CNN Deep Sparse Learning (DSL) module for fine-grained feature extraction.
• Hierarchically fuse detailed and semantic features using an image reconstruction-like decoder.

Main Results:

• CoT achieves competitive performance on PASCAL Context (57.21% mIoU), ADE20K (54.16% mIoU), and Cityscapes (84.23% mIoU).
• Demonstrates robustness against various corruption types in validation studies.

Conclusions:

• The proposed CoT framework effectively integrates semantic and detailed features for superior hierarchical semantic segmentation.
• CoT offers a promising approach for tasks requiring both broad context and fine-grained detail in image analysis.