Mapping the landscape of social behavior

Social interaction is integral to animal behavior. However, lacking tools to describe it in quantitative and rigorous ways has limited our understanding of its structure, underlying principles, and the neuropsychiatric disorders, like autism, that perturb it. Here, we present a technique for high-resolution 3D tracking of postural dynamics and social touch in freely interacting animals, solving the challenging subject occlusion and part-assignment problems using 3D geometric reasoning, graph neural networks, and semi-supervised learning. We collected over 110 million 3D pose samples in interacting rats and mice, including seven monogenic autism rat lines. Using a multi-scale embedding approach, we identified a rich landscape of stereotyped actions, interactions, synchrony, and body contacts. This high-resolution phenotyping revealed a spectrum of changes in autism models and in response to amphetamine not resolved by conventional measurements. Our framework and large library of interactions will facilitate studies of social behaviors and their neurobiological underpinnings.