Mutual information and the encoding of contingency tables

Area of Science:

• Information theory
• Machine learning
• Data analysis

Background:

• Mutual information is a standard metric for comparing object labelings in classification and community detection.
• Conventional mutual information calculations can be biased due to neglecting the information cost of contingency tables.
• Reduced mutual information aims to correct this bias but relies on accurate estimation of information cost bounds.

Purpose of the Study:

• To address the bias in mutual information calculations for comparing labelings.
• To develop an improved method for encoding contingency tables to obtain better bounds on information cost.
• To enhance the accuracy of reduced mutual information as a similarity measure.

Main Methods:

• Developed a novel encoding method for contingency tables.
• Implemented and tested the improved encoding method against conventional approaches.
• Conducted extensive numerical simulations to evaluate the performance.

Main Results:

• The improved encoding method provides a substantially better bound on information cost in typical scenarios.
• The enhanced reduced mutual information approaches the ideal value when labelings are closely similar.
• Numerical results demonstrate the superiority of the proposed method.

Conclusions:

• The new contingency table encoding method significantly improves the accuracy of reduced mutual information.
• This advancement offers a more reliable measure of similarity for competing labelings.
• The findings are crucial for applications relying on accurate classification and community detection performance quantification.