Renormalized Mutual Information for Artificial Scientific Discovery.

Area of Science:

• Information theory
• Machine learning
• Statistical physics

Background:

• Estimating dependence between continuous random variables is crucial for data analysis.
• Existing methods struggle when one variable deterministically depends on another, a common scenario in feature extraction.
• Discovering collective variables and analyzing information flow in neural networks are key challenges.

Purpose of the Study:

• To develop a robust method for estimating mutual information between continuous random variables.
• To address the challenge of deterministic relationships in dependence estimation.
• To provide a tool for feature extraction and understanding complex systems.

Main Methods:

• Derivation of a renormalized mutual information measure.
• Application to scenarios with deterministic dependencies between continuous random variables.
• Utilizing the method for feature extraction and system analysis.

Main Results:

• A well-defined renormalized mutual information was derived.
• The method successfully estimates dependence even with deterministic relationships.
• The approach facilitates the discovery of collective variables and analysis of information flow.

Conclusions:

• The renormalized mutual information offers a powerful tool for analyzing complex dependencies.
• This method enhances artificial scientific discovery by enabling collective variable identification.
• It improves the analysis of information flow in artificial neural networks.