A study on rule extraction from several combined neural networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Extracting rules from neural networks is a computationally challenging problem, often classified as NP-hard.
• Existing methods for rule extraction can be inefficient and may not guarantee high fidelity.
• Deep Interpretable Multilayer Perceptron (DIMLP) networks offer a potential for more transparent AI models.

Purpose of the Study:

• To present a novel, polynomial-time technique for extracting "if-then-else" rules from ensembles of DIMLP neural networks.
• To evaluate the efficiency and accuracy of the proposed rule extraction method.
• To compare the performance of extracted rules against those generated by traditional decision tree algorithms.

Main Methods:

• Development of a new algorithm for rule extraction from DIMLP neural network ensembles.
• The algorithm operates in polynomial time relative to problem dimensionality, data size, and network complexity.
• Validation using four publicly available datasets.

Main Results:

• The proposed method successfully extracts "if-then-else" rules with 100% matching accuracy to the DIMLP network's responses.
• Rules extracted from DIMLP ensembles demonstrated significantly higher accuracy compared to rules extracted from C4.5 decision trees on average.
• The extraction process is computationally efficient, completing in polynomial time.

Conclusions:

• The novel rule extraction technique provides an efficient and highly accurate method for interpreting DIMLP neural network ensembles.
• This approach offers a significant improvement over existing decision tree-based rule extraction methods.
• The findings contribute to enhancing the explainability and practical application of deep learning models.