Multiclass Linear Perceptrons With Multiplicative Margins

Area of Science:

• Machine Learning
• Classification Algorithms
• Pattern Recognition

Background:

• Standard perceptrons lack a margin mechanism, potentially leading to lower classification confidence.
• Additive margin mechanisms can be sensitive to data and weight vector magnitudes.

Purpose of the Study:

• Introduce a novel family of multiclass linear perceptron classifiers, Multiplicative Margin Perceptron (MMPerc).
• Provide an alternative to margin-free and additive margin perceptrons with improved classification confidence.

Main Methods:

• Propose architectural and algorithmic variants of MMPerc.
• Derive loss functions and mistake bounds for separable and nonseparable data.
• Analyze design considerations: bias, margin threshold, and training modes.

Main Results:

• MMPerc classifiers demonstrate superior performance compared to standard perceptrons.
• Experiments show MMPerc outperforms Support Vector Machines and ridge classifiers on synthetic and real datasets.
• The multiplicative margin avoids dependence on score magnitudes.

Conclusions:

• MMPerc classifiers offer simplicity, computational efficiency, and minimalistic design.
• Promising for conventional machine learning, linear evaluation of deep networks, and resource-constrained applications.
• Suitable for integration with hyperdimensional computing and vector symbolic architectures.