Location Property of Convolutional Neural Networks for Image Classification

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Convolutional Neural Networks (CNNs) are pivotal for image classification, with feature representation learning being their core task.
• The effectiveness of CNNs is linked to the quality of feature embeddings in the layer before the softmax operation (BSM-layer).
• A key property for effective feature representation is the distance of embeddings to the separating hyperplane, termed 'Location Property'.

Purpose of the Study:

• To define and leverage the 'Location Property' for optimizing CNN feature representation.
• To introduce a novel loss-based optimization framework, Location Property Loss (LP-loss), for simultaneous optimization of feature embedding directions.
• To develop a new loss function, Secondary Optimal Feature Plane (S-OFP) loss, for the secondary embedding direction.

Main Methods:

• Proposed two feature embedding directions: Principal Embedding Direction (PE-direction) and Secondary Embedding Direction (SE-direction).
• Introduced LP-loss, comprising LP_PE and LP_SE, to guide feature representations.
• Developed S-OFP loss as an optimal solution for the SE-direction, complementing existing losses like softmax, L-softmax, and AM-softmax for the PE-direction.

Main Results:

• Experimental validation on MNIST, SVHN, CIFAR-10, and CIFAR-100 datasets using shallow, moderate, and deep CNN models.
• Demonstrated significant performance improvements when using LP-loss with combinations such as LP_softmax + S-OFP and LP_AM-softmax + S-OFP.
• The results confirm the effectiveness of the 'Location Property' optimization framework in enhancing CNN performance.

Conclusions:

• The proposed LP-loss framework, incorporating S-OFP loss, effectively optimizes CNN feature representations by enhancing the 'Location Property'.
• This approach leads to substantial performance gains across various CNN architectures and benchmark datasets.
• The study validates the importance of feature embedding location and direction in achieving superior image classification results.