Rethinking the word frequency effect: the neglected role of distributional information in lexical processing

Attempts to quantify lexical variation have produced a large number of theoretical and empirical constructs, such as Word Frequency, Concreteness, and Ambiguity, which have been claimed to predict between-word differences in lexical processing behavior. Models of word recognition that have been developed to account for the effects of these variables have typically lacked adequate semantic representations, and have dealt with words as if they exist in isolation from their environment. We present a new dimension of lexical variation that is addressed to this concern. Contextual Distinctiveness (CD), a corpus-derived summary measure of the frequency distribution of the contexts in which a word occurs, is naturally compatible with contextual theories of semantic representation and meaning. Experiment 1 demonstrates that CD is a significantly better predictor of lexical decision latencies than occurrence frequency, suggesting that CD is the more psychologically relevant variable. We additionally explore the relationship between CD and six subjectively-defined measures: Concreteness, Context Availability, Number of Contexts, Ambiguity, Age of Acquisition and Familiarity and find CD to be reliably related to Ambiguity only. We argue for the priority of immediate context in determining the representation and processing of language.