What Is Redundancy?

Area of Science:

• Multivariate Information Theory
• Computational Neuroscience
• Machine Learning

Background:

• The concept of redundancy in multivariate information theory is ambiguously defined, leading to confusion.
• Existing literature conflates operational and informational redundancy, causing paradoxes, especially with independent inputs.

Purpose of the Study:

• To formally distinguish between operational and informational redundancy.
• To resolve conceptual ambiguities and apparent paradoxes in information theory.
• To provide a clear framework for future research on redundancy.

Main Methods:

• Formalizing two distinct classes of redundancy: operational and informational.
• Analyzing functional examples and biased input ensembles.
• Demonstrating distinctions using information theory principles.

Main Results:

• Operational redundancy relates to input sufficiency/substitutability for prediction.
• Informational redundancy concerns shared content among inputs, measured by mutual information.
• Demonstrated that inputs lacking informational overlap can exhibit operational redundancy.

Conclusions:

• A clear separation of operational and informational redundancy is proposed.
• This distinction clarifies why redundancy is elusive and why single measures are insufficient.
• Outlines minimal commitments for future theoretical and applied work.