Distributed Min-Max Learning Scheme for Neural Networks With Applications to High-Dimensional Classification

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• High-dimensional data presents significant challenges for traditional classification algorithms.
• Existing methods often struggle with the curse of dimensionality and feature selection.
• The need for efficient and scalable classification techniques is critical.

Purpose of the Study:

• To introduce a novel learning methodology for classification tasks involving high-dimensional data.
• To address the challenges posed by high-dimensional datasets by employing a game theory framework.
• To develop a method for estimating optimal sparsity in deep neural networks.

Main Methods:

• Formulation of a L1 regularized zero-sum game between penalty coefficients and deep neural network weights.
• Proposal of a distributed learning methodology using additional variables for layerwise cost functions.
• Application of an alternating minimization approach to find the Nash solution for optimal sparsity.

Main Results:

• The Nash solution effectively provides optimal sparsity and classifier compensation.
• A novel computational algorithm enables parallel and distributed implementation.
• Empirical validation on nine datasets demonstrates the approach's efficiency.

Conclusions:

• The proposed distributed learning methodology offers an effective solution for high-dimensional data classification.
• The game theory approach successfully integrates optimal sparsity estimation with deep neural network training.
• The method is computationally efficient and scalable for practical applications.