A model for single and multiple knowledge based networks

The inherent black-box nature of neural networks is an important drawback with respect to the problem of explanation of neural network responses. Although several articles have tackled the problem of rule extraction from a single neural network, just a few papers have investigated rule extraction from several combined neural networks. In this article we describe how to translate symbolic rules into the Discretized Interpretable Multi-Layer Perceptron (DIMLP) and how to extract rules from one or several combined neural networks. Our approach consists of characterizing discriminant hyperplane frontiers. Unordered rules are extracted in polynomial time with respect to the size of the problem and the size of the network. Moreover, the degree of matching between extracted rules and neural network responses is 100% on training examples. We applied single DIMLP networks to 17 data sets related to medical diagnosis and medical prognosis problems. Results based on 10-fold cross-validation showed that the DIMLP model was on average as accurate as standard multi-layer perceptrons (MLP). Furthermore, DIMLP networks were significantly more accurate than CN2 on eight problems, whereas only on one problem CN2 was better than DIMLP. Finally, a non-Hodgkin lymphoma diagnosis problem based on classification of electrophoresis gels was defined. It turned out that ensembles of DIMLP networks were significantly more accurate than CN2 (96.1% +/- 1.4 versus 82.7% +/- 4.0). Finally, symbolic rules revealed the presence of five important spots for the discrimination of the class of Lymphocyte Leukemia/Chronic Lymphoid Leukemia (Lc/LLc), and the class of Centrocytic Lymphoma (Cc).