Two-phase construction of multilayer perceptrons using information theory.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Multilayer perceptrons (MLPs) are widely used neural networks.
• Network pruning is crucial for optimizing MLP performance and efficiency.
• Existing pruning methods have limitations in identifying and removing redundant units.

Purpose of the Study:

• To present a novel two-phase construction approach for pruning MLPs.
• To utilize mutual information (MI) for identifying and removing irrelevant input and hidden units.
• To enhance MLP performance and efficiency through optimized network architecture.

Main Methods:

• A two-phase pruning strategy based on mutual information (MI).
• Phase 1: Ranking input features by relevance to target outputs and eliminating irrelevant ones.
• Phase 2: Sequentially removing redundant hidden units using a novel relevance measure.

Main Results:

• The proposed MI-based pruning strategy demonstrates superior performance compared to related work.
• Experimental results indicate the method is comparable or superior to Support Vector Machine (SVM) and Support Vector Regression (SVR).
• The advantages of the MI-based method are highlighted in comparison to the sensitivity analysis (SA)-based method.

Conclusions:

• The proposed two-phase MI-based pruning approach effectively optimizes MLPs.
• The method offers a competitive alternative to existing techniques like SVM and SVR.
• This strategy provides a valuable tool for developing more efficient and performant neural networks.