Expected energy-based restricted Boltzmann machine for classification.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Restricted Boltzmann Machines (RBMs) are typically used for initial stages in classification.
• Previous methods like Free-Energy based RBM (FE-RBM) have shown promise but can be improved.
• RBMs have been primarily used for feature extraction or network initialization.

Purpose of the Study:

• To propose a self-contained Restricted Boltzmann Machine (RBM) method for classification.
• To introduce an improved RBM approach, the Expected Energy RBM (EE-RBM), for enhanced classification performance.
• To develop a deep learning architecture using stacked EE-RBMs with enhanced connectivity.

Main Methods:

• Proposed a discriminative learning approach for RBMs inspired by FE-RBM.
• Implemented classification using the negative expected energy (EE-RBM) instead of negative free energy.
• Developed a deep learning architecture by stacking EE-RBMs and connecting class nodes to all hidden layers.
• Utilized stochastic gradient descent with a mean-squared error objective for learning.

Main Results:

• EE-RBM achieved competitive classification performance on MNIST and NORB datasets.
• Demonstrated that EE-RBM with binary input nodes performs well in continuous input domains (NORB dataset).
• The proposed deep architecture with enhanced connections further improved classification performance.

Conclusions:

• EE-RBM offers a robust and improved self-contained method for classification tasks.
• The novel deep learning architecture enhances the capabilities of RBMs for complex classification problems.
• EE-RBM provides a competitive alternative to existing classifiers like neural networks and support vector machines.