Large Language Models Persuade Without Planning Theory of Mind

A growing body of work attempts to evaluate the theory of mind (ToM) abilities of humans and large language models (LLMs) using static, noninteractive question-and-answer benchmarks. However, theoretical work in the field suggests that first-personal interaction is a crucial part of ToM and that such predictive, spectatorial tasks may fail to evaluate it. We address this gap with a novel ToM task that requires an agent to persuade a target to choose one of three policy proposals by strategically revealing information. Success depends on a persuader's sensitivity to a given target's knowledge states (what the target knows about the policies) and motivational states (how much the target values different outcomes). We varied whether these states were Revealed to persuaders or Hidden, in which case persuaders had to inquire about or infer them. In Experiment 1, participants persuaded a bot programmed to make only rational inferences. OpenAI's reasoning model o3 excelled in the Revealed condition but performed no better than chance in the Hidden condition, suggesting difficulty with the multistep planning required to elicit and use mental state information. Humans performed moderately well in both conditions, indicating an ability to engage such planning. In Experiment 2, where a human target role-played the bot, and in Experiment 3, where we measured whether human targets' real beliefs changed, o3 outperformed human persuaders, significantly so in the Revealed conditions. These results suggest that effective persuasion can occur without explicit ToM reasoning (e.g., through rhetorical strategies) and that o3 excels at this form of persuasion. Overall, our results caution against attributing human-like PToM to LLMs based on predictive, spectatorial benchmarks while highlighting their potential to influence people's beliefs and behavior.