Related Experiment Videos
Collective mutual information maximization to unify passive and positive approaches for improving interpretation and
1IT Education Center and Graduate School of Science and Technology, Tokai University 4-1-1 Kitakaname, Hiratsuka, Kanagawa 259-1292, Japan.
Summary
This study introduces a simplified method for mutual information maximization in neural networks. The new approach enhances model interpretation and generalization by improving how neurons represent data patterns.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Information Theory
Background:
- Neural network training requires balancing broad input consideration with specific pattern representation.
- Maximizing mutual information between neurons and inputs aids interpretation and generalization.
- Existing methods for mutual information maximization are computationally complex and difficult to apply to large-scale problems.
Purpose of the Study:
- To propose a simplified computational method for mutual information maximization.
- To improve the interpretability and generalization capabilities of neural networks.
Main Methods:
- A novel computational approach for mutual information maximization is presented.
- The method incorporates multiple neural networks to simplify mutual information definition.
- Learning is streamlined using indirect, independent, and fast learning techniques.
Main Results:
- The proposed method successfully increased mutual information on the Australian credit and on-line popularity datasets.
- Mutual information maximization correlated with improved generalization performance.
- Enhanced interpretation performance was observed, attributed to simpler internal representations.
Conclusions:
- The novel method offers a practical and simplified approach to mutual information maximization.
- This technique effectively enhances neural network performance in terms of generalization and interpretation.
- The findings suggest a pathway for more interpretable and effective deep learning models.