Intrinsically motivated humans and agents in open-world exploration

What drives exploration? Understanding the intrinsic motivation to explore is a long-standing challenge in both cognitive science and AI. In this study, we directly compare adults, children aged 8 to 12 y, and AI agents exploring a complex, open-ended environment called Crafter (1). We study how three common information-theoretic intrinsic objectives-entropy, information gain, and empowerment-relate to their behavior. We find that all three are positively correlated with human exploration progress for both children and adults, although entropy and empowerment are more strongly related to exploration progress in children than information gain. Furthermore, across agents and humans we observe that entropy initially increases rapidly, then plateaus, while empowerment increases continuously, suggesting that in early exploration they look for novel states, while in advanced exploration they look for states they can control. Finally, we find preliminary evidence that private speech about goals may aid exploration in children. Our data are available at https://github.com/alyd/humans_in_crafter_data.