Related Experiment Video
Updated: Nov 27, 2025
10:56
Polysome Fractionation and Analysis of Mammalian Translatomes on a Genome-wide Scale
Published on: May 17, 2014
69.3K
From Boltzmann to Zipf through Shannon and Jaynes.
Álvaro Corral1,2,3,4, Montserrat García Del Muro5,6
1Centre de Recerca Matemàtica, Edifici C, Campus Bellaterra, E-08193 Barcelona, Spain.
Entropy (Basel, Switzerland)
|December 8, 2020
Summary
This study uses statistical physics to model word frequencies, finding that letter interactions largely explain word probabilities and Zipf's law. However, individual word probabilities show scattering, indicating limitations of the pure statistical model.
Area of Science:
- Computational Linguistics
- Statistical Physics
- Natural Language Processing
Background:
- Word-frequency distributions in natural language approximate Zipf's law.
- Previous work interprets word frequency via letter interaction potentials.
Purpose of the Study:
- To extend the statistical physics framework for word probabilities.
- To analyze word frequencies up to six letters using the Standardized Project Gutenberg Corpus.
Main Methods:
- Applied Jaynes' maximum-entropy principle with two-letter marginal distribution constraints.
- Utilized the improved iterative-scaling algorithm to determine letter interaction potentials.
- Modeled word probabilities using a Boltzmann distribution based on these potentials.
Main Results:
- The statistical physics model successfully reproduces the general power-law regime of Zipf's law.
- Significant scattering was observed in the probabilities of individual words.
- Empirical two-letter marginal distributions and interaction potentials follow statistical laws.
Conclusions:
- A statistical physics framework can describe word probabilities, aligning with Zipf's law.
- Limitations exist, as individual word probability scattering suggests factors beyond pairwise letter interactions.
- The study provides insights into the statistical regularities governing language structure.
Related Concept Videos
01:20Maxwell-Boltzmann Distribution: Problem Solving
2.1K
Individual molecules in a gas move in random directions, but a gas containing numerous molecules has a predictable distribution of molecular speeds, which is known as the Maxwell-Boltzmann distribution, f(v).
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
2.1K
01:20Relation of DFT to z-Transform
642
The Discrete Fourier Transform (DFT) is a crucial tool for analyzing the frequency content of discrete-time signals. It converts a sequence of N samples from the time domain into its corresponding sequence in the frequency domain, where each sample represents a specific frequency component.
To understand how the DFT works, it's helpful to consider the z-transform, which is a method for representing discrete sequences in the complex frequency domain. The z-transform involves summing the...
To understand how the DFT works, it's helpful to consider the z-transform, which is a method for representing discrete sequences in the complex frequency domain. The z-transform involves summing the...
642
01:27Distribution of Molecular Speeds
4.5K
The motion of molecules in a gas is random in magnitude and direction for individual molecules, but a gas of many molecules has a predictable distribution of molecular speeds. This predictable distribution of molecular speeds is known as the Maxwell-Boltzmann distribution. The distribution of molecular speeds in liquids is comparable to that of gases but not identical and can help to understand the phenomenon of the boiling and vapor pressure of a liquid. Consider that a molecule requires a...
4.5K
01:23Maxwell's Thermodynamic Relations
4.0K
Maxwell's thermodynamic relations are very useful in solving problems in thermodynamics. Each of Maxwell's relations relates a partial differential between quantities that can be hard to measure experimentally to a partial differential between quantities that can be easily measured. These relations are a set of equations derivable from the symmetry of the second derivatives and the thermodynamic potentials.
All thermodynamic potentials are exact differentials. Therefore, their second-order...
All thermodynamic potentials are exact differentials. Therefore, their second-order...
4.0K
02:39Entropy
33.6K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
33.6K
01:18Entropy
3.2K
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
3.2K