DPL++: Advancing the Network Performance via Image and Label Perturbations.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Supervised learning advances often focus on regularizations, optimizers, and architectures.
• Optimizing data distributions and supervisory signals simultaneously for training samples is underexplored.
• Existing methods like Deep Perturbation Learning (DPL) use image perturbations for data distribution rectification but neglect supervisory signals.

Purpose of the Study:

• To introduce a novel paradigm, Deep Perturbation Learning++ (DPL++), for enhancing model generalizability.
• To address the limitations of DPL by synchronously optimizing image and label perturbations.
• To improve training effectiveness and reduce computational complexity compared to previous methods.

Main Methods:

• Developed DPL++ with a synchronous optimization process for both image and label perturbations.
• Formulated differentiable objectives shared between image and label perturbation optimization.
• Applied DPL++ to various backbone architectures (ResNet, DenseNet, ViT) and downstream tasks (image classification, object detection).

Main Results:

• DPL++ significantly enhances model generalizability across various benchmarks by amending data distributions and supervisory signals.
• The synchronous optimization in DPL++ reduces computational complexity, achieving superior performance with fewer iterations compared to DPL.
• Extensive experiments on 2 visual tasks, 5 benchmarks, and 13 backbone networks validate DPL++'s superiority.

Conclusions:

• DPL++ offers a generic and flexible approach to improve supervised learning models.
• The method demonstrates promising capabilities in advancing decision-making, minimizing risk, enhancing class distinguishability, and accelerating training convergence.
• DPL++ represents a significant advancement over DPL, offering a better performance-cost trade-off.