Hierarchical Self-Distilled Feature Learning for Fine-Grained Visual Categorization.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Fine-grained visual categorization (FGVC) requires detailed features from shallow convolutional neural network (CNN) layers.
• Gradient vanishing during backpropagation often hinders the optimization of these vital shallow features.
• Existing methods struggle to effectively leverage shallow features for distinguishing closely related objects.

Purpose of the Study:

• To introduce Hierarchical Self-Distillation (HSD) for generating well-optimized CNN features for accurate FGVC.
• To address the limitations of hard labels in intermediate supervision for FGVC.
• To improve the optimization of shallow layer features crucial for subtle object differentiation.

Main Methods:

• Implemented Hierarchical Self-Distillation (HSD) with multiple intermediate losses to reinforce gradients.
• Utilized soft predictions from deeper network layers for hierarchical self-supervision of shallow layers.
• Introduced Self-Information Entropy Loss (SIELoss) to adaptively soften intermediate predictions.
• Incorporated a Gradient Detached Fusion (GDF) module for multiscale feature fusion.

Main Results:

• HSD significantly improved performance on four challenging fine-grained datasets.
• The GDF module enhanced feature fusion and classification accuracy.
• The proposed methods achieved state-of-the-art classification performance with negligible parameter increase.
• HSD demonstrated superior optimization of shallow CNN features compared to traditional methods.

Conclusions:

• Hierarchical Self-Distillation (HSD) offers an effective approach for optimizing deep CNN features in FGVC.
• The combination of HSD and GDF provides substantial performance gains for fine-grained object recognition.
• This framework presents a promising direction for advancing the accuracy and efficiency of visual categorization tasks.