Deep Classification with Linearity-Enhanced Logits to Softmax Function.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep classification tasks, including image recognition and target detection, are rapidly advancing.
• Convolutional Neural Networks (CNNs) are central to these advancements, with softmax being a key component for performance.
• Existing softmax methods may have limitations in feature discriminability and class separation.

Purpose of the Study:

• To introduce Orthogonal-Softmax, a new, intuitive learning objective function for deep classification tasks.
• To enhance the discriminative power of features extracted by CNNs.
• To simultaneously improve intra-class compactness and inter-class discrepancy.

Main Methods:

• Development of Orthogonal-Softmax, a novel loss function based on Gram-Schmidt orthogonalization for linear approximation.
• Utilizing orthogonal polynomials expansion to establish a stronger relationship compared to traditional and Taylor-Softmax.
• Designing a linear softmax loss to optimize class separation and feature compactness.

Main Results:

• Orthogonal-Softmax demonstrates a stronger relationship via orthogonal polynomials expansion compared to traditional and Taylor-Softmax.
• The proposed loss function effectively acquires highly discriminative features for classification.
• Experiments on four benchmark datasets validate the presented method's effectiveness in promoting intra-class compactness and inter-class discrepancy.

Conclusions:

• Orthogonal-Softmax offers a promising approach to enhance deep classification performance in CNNs.
• The method successfully improves feature discriminability, leading to better classification outcomes.
• Future work may explore the application of Orthogonal-Softmax to non-ground truth samples.