DiSMix: Dimensional Swap Mix for Feature-Level Data Augmentation in Vision Transformers.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Mixup is a data augmentation technique that enhances classification accuracy, especially with limited data or during fine-tuning.
• However, standard mixup can create unnatural representations, potentially hindering fine-tuning performance for models like vision transformers (ViTs).

Purpose of the Study:

• To introduce a novel ViT-aware mixup strategy, Dimensional Swap Mix (DiSMix), to address limitations of conventional mixup methods.
• To improve representation-level data augmentation for fine-tuning vision transformers.

Main Methods:

• DiSMix divides representation vectors into two subspaces.
• It generates new representations by swapping segments between samples and concatenating them.
• This method preserves parts of the original representation, allowing models to learn from partially intact features.

Main Results:

• DiSMix was evaluated by fine-tuning ViTs on the VTAB-1k benchmark.
• It achieved an accuracy of 80.0% on the VTAB-1k Natural split, outperforming conventional mixup methods.
• The results indicate DiSMix's effectiveness in improving fine-tuning performance.

Conclusions:

• DiSMix offers an effective alternative for representation-level data augmentation in fine-tuning scenarios.
• This ViT-aware approach enhances model accuracy by preserving essential features during augmentation.