Optimal layer selection for latent data augmentation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Data augmentation (DA) typically modifies input data.
• Applying DA to hidden layers (feature augmentation) shows potential for performance gains.
• Previous methods for selecting DA layers lacked systematic investigation and were often arbitrary.

Purpose of the Study:

• To investigate optimal layer selection for feature augmentation across diverse experimental settings.
• To develop an automated method for adaptive layer selection in DA.
• To enhance neural network performance through strategic feature augmentation.

Main Methods:

• Systematic investigation of feature augmentation across various training regimes (from scratch, transfer learning), datasets, and models.
• Proposal of the adaptive layer selection (AdaLASE) method.
• AdaLASE utilizes gradient descent to dynamically adjust the ratio of DA application per layer during training.

Main Results:

• Identified trends in suitable layers for feature augmentation based on experimental configurations.
• The proposed AdaLASE method successfully adapted the DA application ratio per layer.
• Achieved high overall test accuracy on multiple image classification datasets.

Conclusions:

• Feature augmentation is a viable strategy for improving neural network performance.
• The AdaLASE method provides an automated and effective approach to optimize feature augmentation layer selection.
• This work offers a more principled and data-driven way to apply DA within neural network architectures.