Information Geometry and Asymptotic Theory for SMML Estimators.

Area of Science:

• Information Theory
• Statistical Modeling
• Computational Geometry

Background:

• Strict Minimum Message Length (SMML) provides a principle for representing continuous statistical models using finite assertions.
• Understanding the geometric and information-theoretic properties of SMML partitions is crucial for efficient data compression and statistical inference.

Purpose of the Study:

• To decompose the SMML objective into assertion entropy and conditional cross-entropy.
• To analyze the geometric structure of SMML partitions in parameter space.
• To establish connections between SMML, Kullback-Leibler (KL) divergence, and Fisher-Rao geometry.

Main Methods:

• Information-theoretic decomposition of the SMML objective.
• Analysis of optimal codepoints using KL divergence minimization.
• Application of Fisher-Rao geometry to derive asymptotic properties of SMML partitions.
• Investigation of SMML in the context of regular canonical exponential families.

Main Results:

• The SMML objective decomposes into assertion entropy and conditional cross-entropy, balancing model selection and data encoding costs.
• Optimal codepoints minimize KL divergence between data and model distributions within partition cells.
• Under a high-resolution regime, SMML partitions are asymptotically equivalent to weighted Fisher-Rao Voronoi tessellations in parameter space.
• SMML codepoints satisfy moment-matching conditions and act as KL/Bregman centroids for exponential families.

Conclusions:

• SMML induces a natural information-geometric quantization.
• The study links entropy-based coding, KL projection, and divergence-based Voronoi geometry.
• SMML provides a principled framework for quantizing statistical models with information-theoretic guarantees.