GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk

Pre-trained large language models (LLMs) have garnered significant attention for their ability to generate human-like textand responses across various domains. This study delves into examines the social and strategic behavior of the commonly used LLM GPT-3.5 by investigating its suggestions in well-established behavioral economics paradigms. Specifically, we focus on socialpreferences, including altruism, reciprocity, and fairness, in the context of two classic economic games: the Dictator Game(DG) and the Ultimatum Game (UG). Our research aims to answer three overarching questions: (1) To what extent do GPT-3.5suggestions reflect human social preferences? (2) How do socio-demographic features of the advisee and (3) technicalparameters of the model influence the suggestions of GPT-3.5? We present detailed empirical evidence from extensiveexperiments with GPT-3.5, analyzing its responses to various game scenarios while manipulating the demographics of theadvisee and the model temperature. Our findings reveal that, in the DG Dictator Game, model suggestions are more altruistic than in humans.We further show that it also picks up on more subtle aspects of human social preferences: fairness and reciprocity. Thisresearch contributes to the ongoing exploration of AI-driven systems' alignment with human behavior and social norms,providing valuable insights into the behavior of pre-trained LLMs and their implications for human-AI interactions.Additionally, our study offers a methodological benchmark for future research examining human-like characteristics andbehaviors in language models.