Accelerate Training of Restricted Boltzmann Machines via Iterative Conditional Maximum Likelihood Estimation.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Restricted Boltzmann Machines (RBMs) are widely used for unsupervised feature learning.
• Training RBMs is challenging due to intractable normalizing constants in their likelihood function.
• Current methods like contrastive divergence use approximations that are slow and can hinder convergence.

Purpose of the Study:

• To develop a novel, efficient algorithm for training Restricted Boltzmann Machines.
• To overcome the limitations of existing gradient approximation methods for RBMs.
• To address the challenge of intractable normalizing constants in RBM likelihood functions.

Main Methods:

• Proposed a fast algorithm for RBM training by treating hidden states as missing data.
• Employed an iterative conditional maximum likelihood estimation approach.
• Developed an extension to handle missing data within RBM training.

Main Results:

• The proposed algorithm significantly outperforms contrastive divergence in RBM training speed and convergence.
• The method effectively avoids the issue of intractable normalizing constants.
• Demonstrated successful application in drug-target interaction prediction.

Conclusions:

• The iterative conditional maximum likelihood approach offers a more efficient and robust method for training RBMs.
• This algorithm provides a substantial improvement over traditional contrastive divergence methods.
• The extension for handling missing data broadens the applicability of RBMs in real-world scenarios like bioinformatics.