Estimation of mutual information for real-valued data with error bars and controlled bias

Area of Science:

• Information theory
• Complex systems analysis
• Statistical inference

Background:

• Estimating mutual information is crucial for analyzing complex, biological, and quantum systems.
• Existing estimators, including the Kraskov et al. method, face limitations in accuracy and applicability.
• Provably optimal estimators for mutual information do not exist, necessitating robust practical solutions.

Purpose of the Study:

• To address the limitations of existing mutual information estimators, particularly the Kraskov et al. method.
• To improve the accuracy, reliability, and applicability of mutual information estimation.
• To provide practical guidelines for using and validating mutual information estimates.

Main Methods:

• Critically evaluating the application of bootstrapping for variance estimation of mutual information.
• Developing an enhanced Kraskov et al. estimator with expanded applicability.
• Implementing methods for bias verification and variance estimation.
• Providing guidance on selecting the free parameter for the estimator.

Main Results:

• Demonstrated pitfalls of naive bootstrapping for mutual information variance estimation.
• Introduced an improved Kraskov et al. estimator with enhanced bias detection and variance estimation.
• Showcased improved performance on synthetic, neurophysiological, and systems biology datasets.
• Provided practical guidelines for parameter selection, enhancing estimator usability.

Conclusions:

• The enhanced mutual information estimator offers improved accuracy and reliability for complex systems.
• The developed methods provide a self-consistent framework for validating mutual information estimates.
• The study offers practical tools and guidelines for researchers in diverse scientific fields.
• The improved estimator is validated across various synthetic and real-world biological datasets.