Probabilistic origins of compositional mental representations

The representation of complex phenomena via combinations of simple discrete features is a hallmark of human cognition. But it is not clear exactly how (or whether) discrete features can effectively represent the complex probabilistic fabric of the environment. This article introduces information-theoretic tools for quantifying the fidelity and efficiency of a featural representation with respect to a probability model. In this framework, a feature or combination of features is "faithful" to the extent that knowing the value of the features reduces uncertainty about the true state of the world. In a single dimension, a discrete feature is faithful if the values of the feature correspond isomorphically to distinct classes in the probability model. But in multiple dimensions, the situation is more complicated: The fidelity of each feature depends on the direction in multidimensional feature space in which the feature is projected from the underlying distribution. More interestingly, distributions may be more effectively represented by combinations of projected features-that is, compositionality. For any given distribution, a variety of compositional forms (features and combination rules) are possible, which can be quite different from one another, entailing different degrees of fidelity, different numbers of features, and even different induced regularities. This article proposes three specific criteria for a compositional representation: fidelity, simplicity, and robustness. The information-theoretic framework introduces a new and potentially useful way to look at the problem of compositionality in human mental representation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).