Association and response accuracy in the wild

We studied contestant accuracy and error in a popular television quiz show, "Jeopardy!" Using vector-based knowledge representations obtained from distributional models of semantic memory, we computed the strength of association between clues and responses in over 5,000 televised games. Such representations have been shown to play a key role in memory and judgment, and consistent with this work, we find that contestants are more likely to provide correct responses when these responses are strongly associated with their clues, and more likely to provide incorrect responses when correct responses are weakly or negatively associated with their clues. This effect is stronger for easier questions with low monetary values, and for questions in which contestants compete to respond quickly. Our results show how distributional models of semantic memory can be used to predict human behavior in naturalistic high-level judgment tasks with skilled participants and significant monetary and social incentives.