Comparing feedforward neural networks using independent component analysis on hidden units.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Neural networks are powerful tools for complex tasks like classification and regression.
• However, their decision-making processes can be opaque, lacking interpretability.
• Existing methods struggle to compare networks with varying architectures or training data.

Purpose of the Study:

• To develop a novel method for comparing feedforward neural networks.
• To assess the functional similarity between different neural network models.
• To enhance the interpretability and understanding of neural network performance.

Main Methods:

• The proposed method utilizes independent component analysis (ICA) on the hidden layers of neural networks.
• It compares pairs of feedforward neural networks, even those with different structures or partially different datasets.
• Experiments were conducted on networks with one hidden layer, varying hidden units, datasets, and activation functions.

Main Results:

• Similar independent components were successfully extracted from compared neural networks, irrespective of structural or data variations.
• Direct comparison of network weights or activations proved insufficient for identifying functional similarities.
• The ICA-based approach effectively revealed resemblances in the internal processing of different neural networks.

Conclusions:

• Independent component analysis provides a robust method for comparing neural networks.
• This technique offers insights into network functionality beyond simple weight or activation comparisons.
• The approach holds potential for improving the understanding and trustworthiness of neural network models.