Triple Spectral Fusion for Sensor-Based Human Activity Recognition

Area of Science:

• Computer Science
• Signal Processing
• Machine Learning

Background:

• Human Activity Recognition (HAR) commonly uses Inertial Measurement Unit (IMU) data for posture, motion, and context.
• Existing learning-based methods struggle with temporal information fusion due to heterogeneous sensor data and long-term context correlation challenges.

Purpose of the Study:

• To propose a novel triple spectral fusion framework for enhanced sensor-based HAR.
• To address the complexities of fusing heterogeneous sensor data and establishing long-term context correlations.

Main Methods:

• Developed an adaptive complementary filtering technique for noise suppression and organized IMU sensor data into modality nodes.
• Applied adaptive filtering in the graph Fourier domain to merge homogeneous and heterogeneous node information within a dynamic heterogeneous graph.
• Implemented adaptive wavelet frequency selection to reduce context redundancy and improve feature length for better temporal aggregation and context correlation.

Main Results:

• The proposed framework effectively fuses multi-sensor data by utilizing adaptive filtering across Fourier, graph Fourier, and wavelet domains.
• Demonstrated superior performance in human activity recognition across ten benchmark datasets.
• Enhanced timestamp-based graph aggregation and long-term context correlation capabilities.

Conclusions:

• The triple spectral fusion framework offers an effective solution for multi-sensor fusion and context correlation in HAR.
• The method shows significant improvements over existing approaches, validated by extensive experimental results.
• The framework advances the field of sensor-based HAR by addressing key temporal fusion challenges.