Adaptive divergence for rapid adversarial optimization.

Area of Science:

• Machine Learning
• Optimization Techniques
• Computational Science

Background:

• Adversarial optimization matches data distributions using divergences, often requiring complex discriminator models.
• High-capacity models offer accurate divergence estimation but need large datasets; low-capacity models are sample-efficient but potentially biased.
• Expensive sampling in complex simulations like parameter tuning increases computational costs.

Purpose of the Study:

• Introduce a novel family of adaptive divergences to accelerate adversarial optimization convergence.
• Reduce the number of samples required from the generator during optimization.
• Improve the efficiency of fine-tuning parameters in complex stochastic simulations.

Main Methods:

• Developed a new family of divergences that dynamically adjust discriminator model capacity during optimization.
• Proposed a strategy where low-capacity models are used for distant distributions, with capacity increasing as distributions converge.
• Applied adaptive divergences to fine-tuning problems using the Pythia event generator and black-box optimization algorithms.

Main Results:

• Demonstrated significant speed-up in optimization convergence, measured by generator samples.
• Adaptive divergences achieved results up to an order of magnitude closer to the optimum compared to Jensen-Shannon divergence within the same budget.
• Validated the effectiveness on parameter fine-tuning tasks for physics simulations.

Conclusions:

• Adaptive divergences offer a substantial acceleration of adversarial optimization, particularly in the early stages.
• This approach is broadly applicable to any stochastic simulation where generator sampling is computationally expensive.
• The dynamic adjustment of model capacity provides a more efficient and effective optimization strategy.