Fixed-Rate Universal Lossy Source Coding and Model Identification: Connection with Zero-Rate Density Estimation and...

Area of Science:

• Information Theory
• Machine Learning
• Data Compression

Background:

• Density estimation with data-rate constraints is crucial for efficient data handling.
• Raginsky's 2008 work established a joint objective for fixed-rate universal lossy source coding and model identification.
• Existing methods often require strong assumptions on density classes.

Purpose of the Study:

• To establish a formal link between constrained density estimation and joint source coding/model identification.
• To derive conditions for achieving Raginsky's joint objective.
• To extend the applicability of universal lossy source coding and model identification to new density classes.

Main Methods:

• Utilizing an equivalent learning formulation.
• Introducing the skeleton estimator, a rate-constrained learning scheme.
• Analyzing the performance in both parametric and non-parametric density settings.

Main Results:

• A necessary and sufficient condition for achievability of the joint objective is derived.
• The skeleton estimator optimally adapts learning parameters for the coding and modeling problem.
• The framework successfully extends to non-parametric L1-totally bounded densities.
• In the parametric case, a prior assumption is removed without performance loss.

Conclusions:

• The skeleton estimator provides a unified framework for rate-constrained density estimation and joint source coding/model identification.
• This work significantly broadens the scope of applicable density classes for these information-theoretic problems.
• The findings offer improved performance and applicability in both parametric and non-parametric scenarios.