AESeg: Affinity-enhanced segmenter using feature class mapping knowledge distillation for efficient RGB-D semantic...

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Dynamic semantic segmentation models offer high accuracy but incur significant computational costs.
• Static segmentation methods, like fully convolutional networks, are computationally efficient but lack global semantic awareness.

Purpose of the Study:

• To develop an efficient semantic segmentation framework that combines the accuracy of dynamic methods with the computational efficiency of static methods.
• To reduce the computational burden associated with learning and inferring class embeddings in dynamic segmentation.

Main Methods:

• Proposed an affinity-enhanced semantic segmentation framework integrating static and dynamic methodologies.
• Constructed a binary affinity matrix encoding pixel-wise category relationships to act as a dynamic classification kernel.
• Introduced a feature-to-category mapping refinement technique for improved accuracy without increased complexity.

Main Results:

• The proposed method achieved performance comparable to purely dynamic approaches with substantially reduced computational overhead.
• Demonstrated state-of-the-art performance on the NYUv2 and SUN-RGBD datasets.
• Verified effectiveness across diverse scenes, including outdoor environments on the CamVid dataset.

Conclusions:

• The affinity-enhanced framework offers a computationally efficient yet accurate solution for semantic segmentation.
• The feature-to-category mapping refinement effectively enhances segmentation accuracy.
• The proposed method represents a significant advancement in deep learning for semantic segmentation.