Geometric Partition Entropy: Coarse-Graining a Continuous State Space

Area of Science:

• Information Theory
• Statistical Mechanics
• Time Series Analysis

Background:

• Traditional entropy estimators (thermodynamic, Shannon) are discrete and problematic for continuous data.
• Differential entropy definitions face limitations similar to thermodynamic challenges.
• Sampled data represents microstates, with underlying macrostates often unknown.

Purpose of the Study:

• Re-examine entropy as a measure of ignorance in continuous phenomena predictability.
• Develop a novel entropy estimator suitable for sampled, continuous data.
• Address limitations of existing discrete and continuous entropy measures.

Main Methods:

• Defined macrostates using data sample quantiles.
• Developed an ignorance density distribution based on quantile distances.
• Calculated geometric partition entropy as the Shannon entropy of this distribution.

Main Results:

• The geometric partition entropy is more consistent and informative than histogram-binning.
• This method excels with complex distributions, outliers, and limited sampling.
• It is computationally efficient and avoids negative values, outperforming k-nearest neighbors estimators.

Conclusions:

• Geometric partition entropy provides a robust quantification of ignorance for continuous phenomena.
• The method is applicable to time series analysis and approximating ergodic symbolic dynamics.
• This estimator offers unique advantages for handling real-world, limited observational data.