Extracting and inserting knowledge into stacked denoising auto-encoders.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks (DNNs) excel at feature learning but suffer from the "black box" problem, hindering real-world applications.
• Interpretability remains a significant challenge for complex DNNs.

Purpose of the Study:

• To propose a novel DNN model, the knowledge-based deep stacked denoising auto-encoder (KBSDAE), that integrates knowledge for improved interpretability and performance.
• To address the limitations of traditional stacked denoising auto-encoders (SDAEs) by incorporating confidence and classification rules.

Main Methods:

• Developed a knowledge discovery algorithm to extract confidence rules for interpreting layerwise denoising auto-encoders (DAEs).
• Created a symbolic language for describing deep networks and representing quantitative reasoning.
• Integrated confidence and classification rules into the deep network structure, specifically the classification layer of SDAEs.

Main Results:

• The proposed KBSDAE model effectively extracts knowledge from deep networks.
• Confidence rule insertion improved feature learning in DAEs.
• Classification rules provided a novel method for knowledge insertion, enhancing SDAE performance.

Conclusions:

• KBSDAE offers better feature learning performance compared to typical DNNs like SDAE.
• The model enhances the understanding of learned representations within deep networks.
• KBSDAE successfully bridges the gap between DNN performance and interpretability.