Information-theoretic evaluation of covariate distributions models

Area of Science:

• Statistical modeling
• Life Sciences
• Data Science

Background:

• Covariate distributions in life sciences often exhibit non-Gaussian margins and complex nonlinear correlations.
• Standard multivariate Gaussian models fail to accurately represent these intricate structures.
• Existing non-Gaussian frameworks like copula models and multiple imputation by chained equations (MICE) lack systematic goodness-of-fit evaluations.

Purpose of the Study:

• To systematically evaluate the goodness-of-fit of non-Gaussian covariate distribution models.
• To compare the performance of copula-based models and MICE against simpler approximations.
• To assess model performance under varying data conditions, including size, dimensionality, and missingness.

Main Methods:

• Utilized Kullback-Leibler (KL) divergence as a scale-invariant goodness-of-fit criterion.
• Developed a novel method for KL divergence confidence intervals using nearest neighbor estimators and subsampling.
• Applied methods to diverse datasets with continuous and discrete covariates, varying in size and dimensionality.

Main Results:

• Non-Gaussian models consistently demonstrated superior KL divergence compared to Gaussian or scale transform approximations.
• KL divergence estimates proved robust to latent variables and substantial missing data.
• Copula models exhibited better generalization to new data, while MICE showed a tendency to overfit.
• Parametric copula models and MICE scaled better with dataset dimensionality than nonparametric copulas.

Conclusions:

• Non-Gaussian models offer significant improvements for modeling realistic life science covariate distributions.
• Copula models are recommended for their generalization capabilities, whereas MICE requires careful evaluation on test data.
• Parametric copula models and MICE provide scalable solutions for high-dimensional datasets.