The Exact VC Dimension of the WiSARD -Tuple Classifier

Area of Science:

• Machine Learning
• Artificial Intelligence
• Neural Networks

Background:

• The Wilkie, Stonham, and Aleksander recognition device (WiSARD) -tuple classifier is a weightless neural network known for single-step learning.
• A key challenge for WiSARD classifiers is RAM node saturation with large datasets, prompting research into mitigation techniques like the bleaching extension.

Purpose of the Study:

• To theoretically determine the exact VC dimension of the basic two-class WiSARD -tuple classifier.
• To determine the exact VC dimension of the bleaching extension of the WiSARD -tuple classifier.
• To analyze the impact of the bleaching technique on the WiSARD classifier's generalization capability.

Main Methods:

• Derivation of the exact VC dimension for the basic WiSARD -tuple classifier.
• Calculation of the exact VC dimension for the WiSARD classifier with the bleaching extension.
• Theoretical analysis comparing the VC dimensions of both models.

Main Results:

• The exact VC dimension of the basic WiSARD -tuple classifier is determined to be linearly proportional to the number of RAM nodes and exponentially related to the tuple length.
• The VC dimension of the bleaching extension is found to be identical to that of the basic WiSARD -tuple classifier.
• Empirical results indicate the bleaching extension achieves high accuracy with low variance.

Conclusions:

• The bleaching technique is confirmed as an enhancement to the WiSARD -tuple classifier, as it does not negatively impact the generalization capability.
• The theoretical findings support the practical effectiveness of the bleaching extension in improving WiSARD classifier performance.