Interpreting Performance of Deep Neural Networks with Partial Information Decomposition

Area of Science:

• Artificial Intelligence
• Machine Learning
• Information Theory

Background:

• Deep neural networks (DNNs) demonstrate limitations in robustness when faced with distributional shifts, hindering real-world deployment.
• Current understanding of how internal information representations within DNNs correlate with robustness is limited.

Purpose of the Study:

• To develop an interpretable framework for assessing DNN robustness using partial information decomposition (PID).
• To quantify the roles of redundancy, uniqueness, and synergy in neural information encoding concerning model robustness.

Main Methods:

• Proposed an interpretable framework based on partial information decomposition (PID).
• Analyzed PID measures (redundancy, unique, synergy) from clean inputs to assess information encoding by neurons.
• Correlated PID measures with model performance under natural corruptions.

Main Results:

• Models with higher redundancy rates and lower synergy rates exhibited more stable performance under various natural corruptions.
• A higher rate of unique information was positively associated with improved classification accuracy on clean data.
• Demonstrated the feasibility of lightweight robustness assessment using internal information analysis.

Conclusions:

• Partial information decomposition offers new insights into understanding and comparing DNN behavior.
• Information-theoretic analysis of internal representations can predict and potentially enhance model robustness without extensive corrupted data.