Neurodynamical classifiers with low model complexity

Area of Science:

• Machine Learning
• Computational Neuroscience
• Statistical Learning Theory

Background:

• The Vapnik-Chervonenkis (VC) dimension is a key metric for measuring model complexity and generalization in machine learning.
• The Minimal Complexity Machine (MCM) offers a novel approach by minimizing an upper bound on the VC dimension.
• Existing methods like Support Vector Machines (SVMs) can be computationally intensive and require numerous support vectors.

Purpose of the Study:

• To develop a neural network that converges to the MCM solution.
• To integrate the MCM neurodynamical system into a broader neural network architecture.
• To optimize both VC dimension bounds and classification error for enhanced model performance.

Main Methods:

• A neural network architecture was designed with the MCM neurodynamical system as its final layer.
• The network's weights across all layers were optimized to minimize a combined objective function.
• The objective function incorporated both a bound on the VC dimension and classification error.
• The model was tested on benchmark datasets from the UCI repository for binary and multi-class classification.

Main Results:

• The proposed neural network successfully converged to the MCM solution.
• The model demonstrated improved generalization capabilities compared to traditional SVMs on benchmark datasets.
• The approach resulted in models utilizing significantly fewer support vectors than SVMs.
• Numerical experiments confirmed the scalability and accuracy of the proposed method.

Conclusions:

• The developed neural network provides an effective method for achieving MCM solutions.
• This approach offers a robust and scalable alternative for binary and multi-class classification tasks.
• The model learns accurate classifiers with improved generalization and reduced complexity, indicated by fewer support vectors.