A Lower Bound on the Differential Entropy of Log-Concave Random Vectors with Applications

Area of Science:

• Information Theory
• Probability Theory
• Convex Geometry

Background:

• Differential entropy quantifies uncertainty in continuous random variables.
• Log-concave distributions are a significant class with desirable properties.
• Rate-distortion theory and channel capacity are fundamental information-theoretic concepts.

Purpose of the Study:

• To derive a lower bound on the differential entropy for log-concave random variables.
• To establish new bounds for the rate-distortion function and channel capacity.
• To generalize these findings to random vectors with dependent coordinates.

Main Methods:

• Derivation of a lower bound on differential entropy using the p-th absolute moment.
• Application of convex geometry tools and techniques.
• Analysis of rate-distortion functions for specific distortion measures and additive noise channels.

Main Results:

• A novel lower bound on differential entropy for log-concave random variables.
• A reverse entropy power inequality with an explicit constant.
• Quantified upper bounds on the difference between the rate-distortion function and Shannon's lower bound (≤ 1.5 bits) and channel capacity and Gaussian channel capacity (≤ 1 bit).

Conclusions:

• The derived bounds provide tighter estimations for information-theoretic measures.
• Results hold for various distortion measures and generalize to multidimensional random variables.
• The findings have implications for source coding, channel coding, and understanding information limits.