Estimating Feature-Label Dependence Using Gini Distance Statistics.

Area of Science:

• Machine Learning
• Statistical Learning Theory
• Data Science

Background:

• Supervised learning fundamentally relies on identifying statistical dependence between features and labels.
• Existing methods like Pearson correlation do not fully characterize independence.
• There is a need for robust dependence measures in machine learning.

Purpose of the Study:

• To present a novel framework for estimating dependence between numerical features and categorical labels.
• To introduce Gini distance-based measures: Gini distance covariance and Gini distance correlation.
• To demonstrate the advantages of Gini distance over existing methods.

Main Methods:

• Utilizing generalized Gini distance, an energy distance within reproducing kernel Hilbert spaces (RKHS).
• Developing Gini distance covariance and Gini distance correlation as dependence measures.
• Deriving uniform convergence bounds and asymptotic bounds for test statistics.

Main Results:

• Gini distance-based measures accurately define both dependence and independence.
• Test statistics are computationally simple and bypass the need for probability density estimation.
• Gini distance statistics exhibit faster uniform convergence than distance covariance statistics.
• Tighter bounds on Type I and Type II errors are achieved with Gini distance.

Conclusions:

• The proposed Gini distance framework provides a superior method for assessing feature-label dependence in supervised learning.
• Gini distance offers improved statistical properties, including faster convergence and more accurate error bounds.
• The method demonstrates strong performance in experimental evaluations.