Topological Knowledge Distillation for Wearable Sensor Data.

Area of Science:

• Wearable sensor technology
• Machine learning applications in healthcare
• Topological data analysis

Background:

• Wearable sensors generate valuable health data, but face challenges with sensitivity, variability, and sampling rates.
• Deep learning models show promise but struggle with wearable data's unique issues.
• Topological Data Analysis (TDA) offers robust feature extraction but faces computational challenges.

Purpose of the Study:

• To develop a compact, robust machine learning model for wearable sensor data analysis.
• To address the computational limitations of TDA in wearable applications.
• To improve the performance of models deployed on edge devices.

Main Methods:

• Proposed a knowledge distillation (KD) strategy using two teacher models: one for raw time-series data and another for topological features (persistence images).
• Implemented an annealing strategy and adaptive temperature for heterogeneous teacher knowledge integration.
• Distilled a compact student model capable of utilizing only time-series data for deployment.

Main Results:

• The distilled student model demonstrated significantly improved performance.
• Incorporating persistence features from the TDA teacher enhanced model accuracy.
• The approach successfully fused deep learning with topological features for effective wearable data analysis.

Conclusions:

• The proposed KD strategy effectively integrates topological features into deep learning models for wearable data.
• This method yields robust, compact models suitable for edge-device deployment.
• This research offers a novel approach to enhance health insights from wearable sensors.