Entropy Gap as a Measure of Epistemic Caution in Credal Sets Generated from Data

Imprecise probability models generated from data represent epistemic uncertainty by replacing the precise empirical distribution with a set of compatible probability distributions. When this set is described by reachable probability intervals, the induced bounds are tight, so the represented imprecision is not inflated by unattainable interval limits. This paper studies the informational effect of this replacement through the epistemic entropy gap, defined as the difference between the maximum entropy over the induced credal set and the Shannon entropy of the empirical distribution. The gap is a differential quantity: it measures the additional uncertainty introduced by the imprecise model beyond the observed frequencies. We analyze it for three reachable interval models generated from multinomial data: the Imprecise Dirichlet Model, the ϵ-contamination model and the approximated Non-Parametric Predictive Inference model. The analysis covers its main properties, its asymptotic behavior and its role in entropy equivalent calibration of model parameters. The results show that the entropy gap offers a common informational scale for comparing how different imprecise models represent the same empirical evidence, and helps interpret the degree of caution associated with limited data reliability and with empirical distributions that may otherwise lead to overconfident uncertainty assessments.