Tiny Machine Learning for Concept Drift.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Tiny machine learning (TML) focuses on deploying machine and deep learning (DL) in resource-constrained embedded systems and IoT devices.
• Existing TML research primarily optimizes model inference, assuming training occurs in cloud/edge environments.
• This assumption leads to TML models becoming obsolete when data-generating processes change (concept drift).

Purpose of the Study:

• To introduce the first Tiny Machine Learning for Concept Drift (TML-CD) solution.
• To enable TML models to adapt to evolving data patterns in real-world applications.
• To ensure the long-term viability and accuracy of TML in pervasive systems.

Main Methods:

• Developed a TML-CD solution integrating deep learning feature extractors and a k-nearest neighbors (k-NNs) classifier.
• Incorporated a hybrid adaptation module for continuous, passive knowledge base updates.
• Implemented an active change detection test (CDT) to identify and remove obsolete knowledge.

Main Results:

• Demonstrated the effectiveness of the TML-CD solution on image and audio benchmarks.
• Validated the TML-CD approach through successful porting on three off-the-shelf micro-controller units (MCUs).
• Showcased the feasibility of TML-CD for real-world pervasive systems.

Conclusions:

• The proposed TML-CD solution effectively addresses concept drift in embedded systems.
• The hybrid adaptation module and CDT enable rapid and continuous model adaptation.
• TML-CD is a practical and feasible approach for maintaining TML model performance in dynamic environments.