Jackknife approach to the estimation of mutual information

Area of Science:

• Statistics
• Data Analysis
• Information Theory

Background:

• Quantifying dependence between random variables is crucial in data analysis.
• Mutual Information (MI) is a key measure, but its estimation from continuous data is problematic.
• Existing kernel estimation methods for MI face challenges with bandwidth selection.

Purpose of the Study:

• To address the challenges in reliably estimating mutual information (MI) from finite continuous data.
• To investigate and improve kernel estimation techniques for MI.
• To provide theoretical underpinnings for the proposed estimation method.

Main Methods:

• Examined kernel estimation of MI, focusing on the equalization of bandwidths.
• Developed a jackknife version of the kernel estimate with equalized bandwidth.
• Allowed bandwidth to vary over an interval for robust estimation.
• Estimated MI using the maximum value across varied bandwidth kernel estimates.

Main Results:

• Demonstrated that bandwidths in kernel estimation of MI should be equalized for improved accuracy.
• Introduced a jackknife approach with equalized bandwidths for MI estimation.
• Established theoretical foundations for the proposed enhanced kernel estimation method.

Conclusions:

• The proposed jackknife kernel estimation with equalized bandwidths offers a more reliable method for estimating mutual information.
• This work provides a significant advancement in the statistical analysis of continuous data.
• The findings contribute to a better understanding of quantifying dependence between random variables.