Deep neural mapping support vector machines.

Area of Science:

• Machine Learning
• Deep Learning
• Computational Intelligence

Background:

• Support Vector Machines (SVMs) performance is highly dependent on kernel choice.
• NEUROSVM architecture integrates multilayer perceptrons for feature extraction with SVMs for classification.
• Traditional kernel methods often rely on implicit functions and kernel tricks.

Purpose of the Study:

• Introduce a novel deep learning model, Deep Neural Mapping Support Vector Machine (DNMSVM), for adaptive kernel learning.
• Develop a method for explicit kernel mapping using neural network sub-networks.
• Enable joint training of the kernel mapping and SVM classifier without kernel tricks.

Main Methods:

• Utilize a sub-network (input and hidden layers) as an explicit kernel mapping function.
• Employ a two-stage training procedure: contrastive divergence learning for sub-network pre-training and gradient descent for joint optimization.
• Implement layer-wise pre-training of the sub-network using Restricted Boltzmann Machines.

Main Results:

• DNMSVM demonstrates superior performance compared to NEUROSVM and Support Vector Machines with Radial Basis Function kernels (RBFSVM).
• The joint training approach in DNMSVM offers advantages over the separate training in NEUROSVM.
• The model effectively learns an adaptive kernel mapping as an explicit function.

Conclusions:

• DNMSVM presents a novel and effective deep learning-based kernel learning method.
• The proposed joint training strategy enhances SVM performance by learning adaptive kernel mappings.
• This approach offers a viable alternative to traditional implicit kernel functions in machine learning.