Can Large Language Models Reason Strategically? Evidence From Attacker-Defender Signaling Games

Area of Science:

• Artificial Intelligence
• Behavioral Economics
• Game Theory

Background:

• Large language models (LLMs) are being considered for strategic decision-making under uncertainty.
• It is unclear if LLM behavior aligns with normative models or human strategic actions in adversarial settings.

Purpose of the Study:

• To evaluate GPT-4o's strategic decision-making in an attacker-defender signaling game.
• To compare GPT-4o's behavior against normative Bayesian benchmarks and empirical human decisions.

Main Methods:

• A controlled attacker-defender signaling game was employed.
• GPT-4o's performance was benchmarked against a normative Bayesian best-response model and human experimental data.
• Strategic behavior was decomposed into belief formation and action selection.

Main Results:

• GPT-4o's modal actions partially aligned with normative predictions but decision distributions significantly diverged.
• GPT-4o systematically underutilized the 'abort' option compared to normative recommendations.
• GPT-4o's behavior did not align with human decision-making patterns.

Conclusions:

• LLMs represent a distinct class of strategic agents, separate from rational equilibrium players and boundedly rational humans.
• A cognitive-action decoupling was observed in GPT-4o, where diffuse beliefs led to deterministic actions.
• Findings have implications for deploying LLMs in high-stakes adversarial roles requiring strategic abstention.