Knowledge Transfer-Based Sparse Deep Belief Network.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks (DNNs) face development bottlenecks due to increasing complexity and the
• black box
• problem.
• Existing deep belief networks (DBNs) require optimization for efficiency and interpretability.

Purpose of the Study:

• To propose a novel deep belief network (DBN) model that enhances knowledge transfer and optimizes network structure.
• To develop a method for extracting rules from deep networks to understand their operational mechanisms.
• To create a sparse DNN with minimal information loss for improved performance.

Main Methods:

• A neural-symbolic model was developed to extract operational rules from deep networks.
• Knowledge fusion techniques, including rule merging and deletion, were applied to the DBN model.
• A knowledge transfer-based sparse DBN (KT-SDBN) was constructed for efficient network generation.

Main Results:

• The KT-SDBN demonstrated a more sparse network structure compared to traditional DBNs.
• Effective feature learning was achieved using only 30% of the original network parameters.
• The KT-SDBN exhibited a significantly higher compression rate than other structure optimization algorithms.

Conclusions:

• KT-SDBN offers improved learning performance and a more parsimonious network structure.
• The proposed method effectively addresses the limitations of DNNs by enabling knowledge transfer and structural optimization.
• KT-SDBN represents a promising advancement in creating efficient and interpretable deep learning models.