Decreasing the Size of the Restricted Boltzmann Machine

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Restricted Boltzmann Machines (RBMs) are generative stochastic artificial neural networks used in deep learning.
• Reducing the number of hidden units in RBMs can lead to model compression and faster inference.
• However, decreasing hidden units often results in a performance degradation.

Purpose of the Study:

• To propose a novel method for decreasing the number of hidden units in Restricted Boltzmann Machines (RBMs).
• To maintain or improve model performance, quantified by Kullback-Leibler divergence, after reducing hidden units.
• To demonstrate the efficacy of the proposed algorithm through numerical simulations.

Main Methods:

• The study proposes a specific algorithm designed to optimize the reduction of hidden units in RBMs.
• Performance is evaluated using Kullback-Leibler divergence as a key metric.
• Numerical simulations are conducted to validate the proposed method.

Main Results:

• The proposed method successfully decreases the number of hidden units in the RBM.
• Performance, as measured by Kullback-Leibler divergence, is maintained despite the reduction in hidden units.
• Numerical simulations confirm the effectiveness of the algorithm.

Conclusions:

• The developed algorithm offers an effective way to create more efficient RBMs.
• This approach allows for model compression without compromising predictive accuracy.
• The findings are supported by empirical evidence from simulations.