REDRESS: Generating Compressed Models for Edge Inference Using Tsetlin Machines

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Edge computing faces trade-offs between resource efficiency (energy, memory) and performance (speed, accuracy) for embedded machine learning models.
• Conventional approaches like Binary Neural Networks (BNNs) often struggle to balance these competing demands.
• Tsetlin Machine (TM), an emerging algorithm using learning automata for propositional logic, offers a potential alternative.

Purpose of the Study:

• To propose a novel algorithm-hardware co-design methodology, REDRESS, for efficient training and inference of Tsetlin Machines (TMs) on resource-constrained edge devices.
• To significantly reduce the memory footprint of TMs through lossless compression and optimized training.
• To achieve high-speed inference without compromising accuracy, targeting low and ultra-low power applications.

Main Methods:

• Developed REDRESS, a methodology featuring independent TM training and inference techniques.
• Introduced 'include-encoding' for lossless compression of Tsetlin Automata (TA), achieving over 99% compression by storing only 'include' information.
• Implemented a computationally minimal training procedure, 'Tsetlin Automata Re-profiling', to enhance accuracy and TA sparsity.
• Designed a bit-parallel inference algorithm operating directly on the compressed TA data, eliminating runtime decompression.

Main Results:

• The REDRESS methodology demonstrated superior performance of Tsetlin Machines (TMs) over state-of-the-art Binary Neural Networks (BNNs) across five benchmark datasets (MNIST, CIFAR2, KWS6, Fashion-MNIST, Kuzushiji-MNIST).
• Achieved over 99% lossless compression for Tsetlin Automata (TA) using the 'include-encoding' method.
• On an STM32F746G-DISCO microcontroller, REDRESS delivered speedups and energy savings ranging from 5x to 5700x compared to various BNN models.
• The proposed training and inference techniques significantly reduced memory footprint and computation time.

Conclusions:

• REDRESS offers a highly efficient approach for deploying Tsetlin Machines (TMs) in edge computing environments, effectively addressing the resource-performance trade-offs.
• The methodology, including lossless compression and bit-parallel inference, enables significant memory savings and substantial speedups compared to existing BNN models.
• REDRESS paves the way for advanced machine learning capabilities in low-power and ultra-low-power embedded systems.