On the Versatile Uses of Partial Distance Correlation in Deep Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Comparing neural network functional behavior is crucial for understanding learning and identifying improvements.
• Existing methods for systematic functional comparison across different networks are limited and often layer-by-layer.
• Techniques like canonical correlation analysis (CCA) are underutilized for this purpose.

Purpose of the Study:

• To introduce and adapt distance correlation, a statistical method, for comparing functional behaviors of neural networks.
• To demonstrate the deployment of distance correlation for large-scale deep learning models.
• To explore novel applications enabled by this comparative analysis.

Main Methods:

• Revisiting and adapting the statistical concept of distance correlation and its partial variant.
• Developing a framework for applying distance correlation to analyze feature spaces of neural networks with different dimensions.
• Implementing the method for large-scale deep learning models.

Main Results:

• Distance correlation provides a systematic way to compare functional behaviors across different neural networks.
• The method enables diverse applications including model conditioning, learning disentangled representations, and optimizing diverse models.
• Experiments suggest distance correlation acts as a versatile regularizer with significant advantages.

Conclusions:

• Distance correlation offers a powerful and versatile tool for analyzing and comparing neural network functional behaviors.
• This approach facilitates a deeper understanding of what neural networks learn and how to improve their efficiency and robustness.
• The method overcomes common challenges in functional analysis of deep learning models.