Integration of affective cues in context-rich and dynamic scenes varies across individuals

Humans need to make rapid and accurate judgments of others' emotions to understand and navigate the social world around them. To do so, humans combine multiple sources of emotional information from facial expressions and contextual information. However, it is not well understood how different sources of information are integrated, let alone how observers assess which signals should be combined. Across three studies (n = 944) using data from new and previously collected datasets, we investigate whether affective inferences follow a Bayesian framework where information is optimally weighted based on its ambiguity and then combined. We compare this model to a more parsimonious Heuristic integration model that averages cues without considering cue ambiguity. We find that the Bayesian model best predicts individual observers' inferences of affect, but there are significant individual differences in integration strategies, with some individual observers adopting a Heuristic strategy. We also find that integration models that use stable weights instead of dynamic weights, as well as non-integration models, fail to predict observers' affective judgments. Our findings suggest that there are significant idiosyncratic differences in how humans combine affective cues, where some observers use a Bayesian framework to weigh individual cues before integration, while others use efficient but less optimal strategies.