Constrained Adaptive Distillation Based on Topological Persistence for Wearable Sensor Data.

Area of Science:

• Machine Learning
• Wearable Sensor Data Analysis
• Topological Data Analysis (TDA)

Background:

• Topological Data Analysis (TDA) extracts persistence features from wearable sensor data, offering valuable insights.
• However, TDA's computational and time resource demands for feature extraction are significant.
• Existing knowledge distillation (KD) methods struggle with knowledge gaps when teachers use different data modalities.

Purpose of the Study:

• To develop a robust knowledge distillation framework for efficient wearable sensor data analysis.
• To effectively integrate multimodal features from different teacher models into a student model.
• To overcome performance limitations caused by knowledge gaps in current KD approaches.

Main Methods:

• Utilized knowledge distillation (KD) with multiple teacher networks trained on raw time-series and TDA-generated persistence images.
• Introduced a multimodal feature integration framework to bridge the knowledge gap between teachers.
• Employed an entropy-based constrained adaptive weighting mechanism to balance teacher contributions and used batch/channel similarities for structural information assimilation.

Main Results:

• The proposed framework effectively integrates multimodal features from diverse teacher models.
• The adaptive weighting mechanism successfully balances teacher influences, enhancing student model learning.
• Demonstrated significant effectiveness of the method on wearable sensor data analysis.

Conclusions:

• The developed KD framework efficiently analyzes wearable sensor data by integrating multimodal features.
• The approach mitigates computational burdens associated with TDA feature extraction.
• This method offers a promising solution for improving performance and efficiency in wearable sensor data applications.