Are artificial agents perceived similarly to humans? Knowns, unknowns and the road ahead

Starting long before modern generative artificial intelligence (AI) tools became available, technological advances in constructing artificial agents spawned investigations into the extent to which interactions with such non-human counterparts bear resemblance to human-human-interactions. Although artificial agents are typically not ascribed a mind of their own in the same sense as humans, several researchers concluded that social presence with or social influence from artificial agents can resemble that seen in interactions with humans in important ways. Here we critically review claims about a comparability between human-agent interactions and human-human-interactions, outlining methodological approaches and challenges which predate the AI era but continue to influence work in the field. By connecting novel work on AI tools with broader research in the field we aim to provide orientation and background knowledge to researchers as they move forward in inquiring how artificial agents are used and perceived, and to further contribute to an ongoing discussion around appropriate experimental setups and measures. We argue that both when confronting participants with simple artificial agents or AI-driven bots, researchers should (1) scrutinize the specificity of measures which may indicate social as well as more general, non-social processes, (2) avoid deceptive cover stories which entail their own complications to data interpretation and (3) see value in understanding specific social-cognitive processes in interactions with artificial agents even when the most generalizable comparisons with human-human interactions may not be achieved in a specific experimental setup.