Uncertainty-aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data

Area of Science:

• Machine Learning
• Wearable Sensor Data Analysis
• Topological Data Analysis

Background:

• Topological data analysis (TDA) features, particularly persistence images (PIs), enhance machine learning for wearable sensor data analysis due to their robustness to perturbations.
• However, generating PIs is computationally expensive, limiting their application on resource-constrained devices.

Purpose of the Study:

• To develop a computationally efficient method for leveraging TDA features in wearable sensor data analysis.
• To create a compact machine learning model that incorporates the benefits of TDA without its computational overhead.

Main Methods:

• Proposed an uncertainty-aware topological persistence guided knowledge distillation (KD) approach.
• Utilized multiple teachers (raw time-series and topological features) to distill knowledge into a single student model.
• Implemented feature harmonization techniques, including separating common/distinct components, weighting, and uncertainty rectification.

Main Results:

• The proposed KD method successfully created a robust single student model operating solely on time-series data at test-time.
• Empirical evaluations across diverse datasets and models demonstrated the robustness and efficacy of the approach.
• Achieved an approximate 4.3% enhancement in classification performance compared to a model trained from scratch on GENEActiv data.

Conclusions:

• The uncertainty-aware topological persistence guided KD method effectively distills complex TDA features into a compact model.
• This approach overcomes the computational challenges of TDA, enabling its practical application in resource-limited wearable sensor scenarios.
• The proposed method offers a significant performance improvement for wearable sensor data classification.