Disentangling contextual diversity: Communicative need as a lexical organizer

Area of Science:

• Cognitive Psychology
• Computational Linguistics
• Psycholinguistics

Background:

• Contextual diversity (CD) is a measure that refines word frequency by accounting for unique contexts.
• Previous research indicates CD better predicts lexical organization than raw word frequency.
• The principle of likely need suggests words used in diverse past contexts are more likely needed in the future.

Purpose of the Study:

• To redefine linguistic context by incorporating socially based measures.
• To investigate if large-scale online communication patterns improve lexical organization models.
• To challenge traditional definitions of linguistic context in cognitive models.

Main Methods:

• Utilized a large dataset (over 55 billion words) from Reddit discussions.
• Derived socially based contextual measures from individual communication patterns.
• Developed count-based and semantic diversity models of contextual diversity.

Main Results:

• Communication patterns across diverse discourses provided the best fit for lexical organization data.
• Socially derived contextual measures outperformed traditional local context definitions.
• This suggests a need to reevaluate how linguistic context influences word strength in the lexicon.

Conclusions:

• Linguistic context should be redefined to include broader social communication patterns.
• The "likely need" principle is better supported by socially inclusive contextual measures.
• Future research on word representation and memory should consider large-scale social interaction data.