Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

Area of Science:

• Computational Economics
• Artificial Intelligence
• Game Theory

Background:

• Firms competing on price in uniform price auctions seek Nash equilibria.
• Deep reinforcement learning (DRL) offers potential for analyzing complex market dynamics.
• Traditional DRL is often 'model-free,' relying on extensive parameter tuning.

Purpose of the Study:

• To evaluate the performance of deep deterministic policy gradient (DDPG) for finding Nash equilibria in price competition.
• To systematically analyze the impact of DDPG parameter configurations on convergence to the Bertrand equilibrium.
• To assess DDPG's applicability in more complex multi-player auction settings.

Main Methods:

• Utilized deep deterministic policy gradient (DDPG), a DRL algorithm for continuous state and action spaces.
• Systematically varied DRL algorithm parameters (learning rates, memory buffers, etc.).
• Compared convergence to analytically derived Bertrand equilibrium in a uniform price auction model.

Main Results:

• Identified DDPG parameter configurations achieving up to 99% convergence rates.
• Demonstrated reliable convergence in settings with multiple players and varied cost structures.
• Validated DDPG's effectiveness beyond simple market models.

Conclusions:

• Optimized DDPG parameter settings enable high-fidelity convergence to Nash equilibria in price competition.
• DRL, specifically DDPG, is a robust tool for studying strategic firm behavior in complex auction environments.
• This approach facilitates the analysis of economic strategies in sophisticated market simulations.