Comparing support vector machines and feedforward neural networks with similar hidden-layer weights.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Support Vector Machines (SVMs) typically require numerous support vectors for output generation.
• Recent advancements aim to create SVMs with fewer basis functions while retaining support vector properties.
• Sequential Feedforward Neural Networks (FNNs) also exhibit sparse model properties with controllable hidden units.

Purpose of the Study:

• To compare the performance of standard Support Vector Machines (SVMs) against sequential Feedforward Neural Networks (FNNs).
• To evaluate both models under identical conditions, focusing on similar hidden-layer weight constraints.
• To analyze model sparsity, accuracy, and computational efficiency.

Main Methods:

• An experimental study was conducted on multiple benchmark datasets.
• Support Vector Machines (SVMs) and sequential Feedforward Neural Networks (FNNs) were implemented.
• Models were trained and evaluated under consistent experimental conditions.

Main Results:

• Accuracy outcomes for both SVMs and sequential FNNs were found to be highly comparable.
• Sequential FNNs generated models with fewer hidden units than standard SVMs, aligning with 'sparse' SVMs.
• SVMs demonstrated lower computational times compared to the sequential FNNs.

Conclusions:

• Sequential FNNs offer a viable alternative to SVMs, achieving similar accuracy with enhanced model sparsity.
• The choice between SVMs and sequential FNNs may depend on the trade-off between computational speed and model size.
• Both approaches, when constrained to similar hidden-layer weights, present competitive performance characteristics.