Multiattentive Perception and Multilayer Transfer Network Using Knowledge Distillation for RGB-D Indoor Scene Parsing

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Scene parsing is crucial in computer vision but current methods often lack efficiency.
• High computational costs and large model sizes hinder real-time applications of existing scene parsing techniques.

Purpose of the Study:

• To develop an efficient and accurate scene parsing network.
• To address the limitations of model parameters and computational size in current methods.

Main Methods:

• Proposed a multiattentive perception and multilayer transfer network employing knowledge distillation (MPMTNet-KD).
• Utilized a student network (MPMTNet-S) guided by a teacher network (MPMTNet-T) with novel multilayer knowledge distillation (KD) methods.
• Introduced a multiattentive perception module (MAPM) and hetero-oriented sensing (HOS) convolution for feature extraction and integration.
• Employed discrete cosine transform (DCT) with filtering to enhance depth information and knowledge transfer.

Main Results:

• MPMTNet-KD significantly reduced parameters from 125.8 M (MPMTNet-T) to 28.3 M (MPMTNet-S).
• Achieved a mean intersection over union (mIoU) of 54.9% on the NYUDv2 indoor scene parsing benchmark.
• Demonstrated generalization capacity on MFNet and PST900 datasets.

Conclusions:

• The proposed MPMTNet-KD offers an efficient solution for real-time scene parsing.
• The multilayer transfer KD approach effectively improves knowledge distillation for computer vision tasks.
• The method enhances depth information processing and achieves competitive performance with reduced computational complexity.