Topological Persistence Guided Knowledge Distillation for Wearable Sensor Data.

Area of Science:

• Wearable sensor data analysis
• Machine learning for health insights
• Time-series data processing

Background:

• Deep learning excels in converting sensor data to health insights but struggles with signal quality and user variability in activity recognition.
• Topological Data Analysis (TDA) offers robust features but faces challenges in computational load and integrating with deep learning representations.
• Existing methods lack efficient fusion of TDA's robustness and deep learning's performance for time-series health data.

Purpose of the Study:

• To develop a novel method for integrating topological features into deep learning models for time-series wearable sensor data.
• To address the computational cost and representation mismatch issues hindering TDA-based deep learning.
• To distill a compact and robust student model capable of implicit topological feature preservation.

Main Methods:

• Proposed a knowledge distillation (KD) framework using two teacher networks: one on raw time-series data, another on TDA-derived persistence images.
• Introduced new constraints, including orthogonality on feature correlation maps, to enhance feature expressiveness and facilitate knowledge transfer.
• Implemented an annealing strategy within KD to accelerate convergence and improve feature accommodation.

Main Results:

• The distilled student model, using only raw time-series data at test-time, achieved 71.74% classification accuracy on GENEActiv data with a 1D CNN.
• The proposed method significantly outperformed baseline approaches in activity recognition accuracy.
• The distilled model demonstrated substantially reduced processing time compared to the teacher networks, processing 6k samples in under 17 seconds.

Conclusions:

• The KD approach effectively integrates the complementary strengths of TDA and deep learning for wearable sensor data analysis.
• The resulting compact student model implicitly preserves topological features, offering a robust and efficient solution for activity recognition.
• This method overcomes key obstacles in applying TDA to deep learning, paving the way for more powerful health monitoring systems.