Playing repeated games with large language models.

Area of Science:

• Artificial Intelligence
• Game Theory
• Computational Social Science

Background:

• Large language models (LLMs) are increasingly integrated into human-interactive systems.
• Understanding their emergent social behaviors, like cooperation and coordination, is crucial.
• Behavioral game theory offers a framework to analyze these interactions.

Purpose of the Study:

• To investigate the cooperation and coordination behaviors of LLMs using behavioral game theory.
• To compare LLM performance against human-like strategies and actual human players.
• To explore methods for improving LLM coordination in interactive scenarios.

Main Methods:

• LLMs were engaged in finitely repeated 2x2 games against other LLMs, human-like strategies, and human players.
• Performance was evaluated across different game types, focusing on self-interested vs. coordination-required scenarios.
• GPT-4's behavior was modulated using opponent information and a 'social chain-of-thought' strategy.

Main Results:

• LLMs demonstrated strong performance in self-interested games (e.g., iterated Prisoner's Dilemma).
• LLMs exhibited suboptimal behavior in coordination games (e.g., Battle of the Sexes).
• Modulating GPT-4's strategy improved its scores and coordination success with human players.

Conclusions:

• LLMs possess distinct behavioral signatures in game-theoretic interactions, excelling in competition but lagging in cooperation.
• The social behavior of LLMs can be influenced by contextual information and strategic prompting.
• This research establishes a foundation for a behavioral game theory tailored for artificial intelligence agents.