Related Experiment Video
Updated: May 14, 2026
05:33
Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning
Published on: January 29, 2020
Entropy's Dilemma: Facilitatory and Inhibitory Effects of Information Entropy During Language Processing
Hossein Karimi1, Jaden Zinn2, Haomiao Li2
1Department of Psychology, California State University, Fullerton.
Cognitive Science
|May 13, 2026
Summary
Predicting multiple words in language processing involves conditional entropy. Higher entropy aids semantic processing but hinders lexical competition, impacting word recognition.
Area of Science:
- Cognitive Psychology
- Psycholinguistics
- Computational Linguistics
Background:
- Language processing relies on predicting upcoming words based on context.
- Conditional entropy quantifies the uncertainty or distribution of plausible next words.
- Previous models did not fully disentangle the dual effects of conditional entropy on lexical processing.
Purpose of the Study:
- To investigate how conditional entropy influences lexical processing during language comprehension.
- To differentiate the effects of semantic feature activation versus lexical competition within conditional entropy.
- To examine the role of predictive processing in language.
Main Methods:
- Participants completed sentence fragments, generating word completions and probabilities.
- Eye movements were tracked as participants read sentences with target words.
- Conditional entropy was calculated, decomposed into semantic overlap and lexical entropy.
Main Results:
- Increased semantic overlap (facilitation) positively impacted lexical processing.
- Increased lexical entropy (competition) negatively impacted lexical processing.
- These findings were replicated in an independent dataset.
Conclusions:
- Conditional entropy's influence on language processing is twofold: semantic activation facilitates, while lexical competition inhibits.
- Decomposing conditional entropy is crucial for understanding its effects.
- Results inform predictive processing theories and the development of large language models.
More Related Videos
05:22Dissociation of the Confounding Influences of Expectancy and Integrative Difficulty Residing in Anomalous Sentences in Event-related Potential Studies
Published on: May 9, 2019
05:38Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology
Published on: June 29, 2021
Related Concept Videos
01:27Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
02:39Entropy
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
01:18Entropy
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
01:14The Second Law of Thermodynamics
In the quest to identify a property that may reliably predict the spontaneity of a process, a promising candidate has been identified: entropy. Scientists refer to the measure of randomness or disorder within a system as entropy. High entropy means high disorder and low energy. To better understand entropy, think of a student’s bedroom. If no energy or work were put into it, the room would quickly become messy. It would exist in a very disordered state, one of high entropy. Energy must be put...
01:20Entropy and the Second Law of Thermodynamics
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
01:26Entropy and the Second Law of Thermodynamics
Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...