Dissecting Deep Learning Networks-Visualizing Mutual Information.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Deep learning (DL) networks are advanced AI models with complex architectures.
• Current understanding of hidden unit roles in DL networks is limited, creating a knowledge gap.
• Evaluating DL performance solely on output accuracy overlooks the contributions of internal network components.

Purpose of the Study:

• To explore visualization techniques for illustrating mutual information (MI) within DL networks.
• To understand the specific roles of DL units in network classification performance.
• To bridge the gap between DL applications and theoretical understanding.

Main Methods:

• Utilized mutual information (MI) as a theoretical measurement for variable relationships.
• Applied visualization techniques to analyze MI patterns in DL networks.
• Conducted experiments on several popular DL network architectures.

Main Results:

• Visualizing MI between input/output and hidden layers/units clarifies unit roles.
• Observed MI change patterns provide insights into network convergence.
• Identified redundancy and less-effective units through MI visualization.

Conclusions:

• MI visualization offers a powerful tool for understanding DL network internals.
• This approach facilitates more objective evaluation and potential improvement of DL network design.
• Enhanced insights can lead to the development of more efficient and effective DL architectures.