Relevance in the Renormalization Group and in Information Theory

Area of Science:

• Theoretical Physics
• Machine Learning
• Information Theory

Background:

• Analyzing complex physical systems requires identifying key degrees of freedom.
• Machine learning offers powerful tools but often lacks interpretability, obscuring connections to physical theory.
• Current methods struggle to link learned features to established physical concepts.

Purpose of the Study:

• To establish a formal link between field-theoretic relevance and information-theoretic relevance.
• To develop a systematic method for interpreting machine learning features in physics.
• To bridge the gap between deep learning and fundamental physical theories.

Main Methods:

• Defined an information-theoretic notion of relevance using the information bottleneck (IB) formalism.
• Established analytical equivalence between IB compression and renormalization group relevance.
• Utilized field theory for statistical physical systems.
• Confirmed theoretical predictions through numerical simulations.

Main Results:

• Demonstrated that IB compression analytically identifies relevant degrees of freedom corresponding to lowest scaling dimensions in field theories.
• Showed that IB solutions are dependent on the physical symmetries of the data.
• Provided a quantitative connection between information theory and renormalization group methods.

Conclusions:

• The study provides a theoretical framework connecting information bottleneck theory and renormalization group relevance.
• This work offers a method to incorporate physical interpretability into deep learning applications in physics.
• Findings facilitate a deeper understanding of complex physical systems through interpretable machine learning.