Related Experiment Video
Updated: Jun 20, 2026
08:12
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
Comparing frameworks for similarity-aware generalized entropy (sentropy)
Phuc Nguyen1, Josiah Couch1, Rahul Bansal1
1Beth Israel Deaconess Medical Center, Department of Pathology, Boston, Massachusetts 02215, USA.
Physical Review. E
|June 19, 2026
Summary
Shannon entropy is insufficient for machine learning datasets. Similarity-sensitive entropy, measured by Leinster-Cobbold-Reeve (LCR) or Vendi Score (VS), captures richer information. LCR is generally preferred for broader applications.
Area of Science:
- Information theory
- Machine learning
- Data analysis
Background:
- Traditional Shannon entropy focuses on element frequencies in datasets.
- It overlooks crucial information embedded in element similarities and differences.
- Similarity-sensitive entropy measures like sentropy address this limitation.
Purpose of the Study:
- To compare the Leinster-Cobbold-Reeve framework (LCR) and Vendi Score (VS) for measuring sentropy.
- To determine which method is more suitable for machine learning datasets.
- To analyze the theoretical and numerical differences between LCR and VS.
Main Methods:
- Theoretical analysis and numerical computation.
- Utilized 53 diverse and large-scale imaging and tabular datasets.
- Introduced and utilized the concept of "half distance" to parameterize similarity scaling.
Main Results:
- LCR and VS values can vary significantly and are often complementary.
- Both metrics' results are dependent on the scaling of similarities.
- VS provides an upper bound for LCR under specific conditions of the Rényi-Hill order parameter.
Conclusions:
- VS is recommended for datasets interpretable as linear combinations of "ur-elements" or with quantum mechanical characteristics.
- LCR is generally favored for capturing rich information from element similarities, differences, and frequencies.
- LCR and VS can be complementary at specific "half distances".
Related Concept Videos
01:26Causes of Similarity-Dissimilarity Effect
The similarity-dissimilarity effect, a fundamental concept in social psychology, explains how interpersonal similarities and differences influence attraction and social interactions. This effect is supported by three key psychological perspectives: balance theory, social comparison theory, and consensual validation.Balance Theory and Cognitive ConsistencyBalance theory, developed by Fritz Heider, posits that individuals seek cognitive consistency in their relationships. When two people share...
02:39Entropy
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
01:18Entropy
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
01:14The Entropy as a State Function
Consider an arbitrary process that moves between two specific states (A and B) in a cyclic manner. This process is reversible and broken down into smaller parts that each follow a Carnot cycle. A Carnot cycle has two isothermal (constant temperature) processes. During these processes, the ratio of the amount of heat transferred to their respective temperature remains constant. The other two processes in the Carnot cycle are also reversible but adiabatic, which means they occur without any heat...
01:20Entropy and the Second Law of Thermodynamics
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
01:26Entropy and the Second Law of Thermodynamics
Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...