Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition

Area of Science:

• Computer Science
• Machine Learning
• Signal Processing

Background:

• Sensor-based human activity recognition (HAR) commonly utilizes convolutional neural networks (CNNs) for local feature extraction from time-series data.
• Transformer models excel at long-range sequence modeling but can overlook local dependencies.
• Integrating CNNs and Transformers offers a promising approach to leverage the strengths of both architectures for enhanced HAR.

Purpose of the Study:

• To propose a novel Dual-branch Interactive Network (DIN) that synergistically combines CNN and Transformer architectures for multi-channel time-series sensor data.
• To develop a lightweight model for human activity recognition (HAR) suitable for resource-constrained wearable devices.
• To improve the accuracy and efficiency of HAR systems by effectively capturing both local and global features.

Main Methods:

• A dual-stream architecture is employed, utilizing conv-embedding for local features and patch-embedding for global features.
• A co-attention mechanism adaptively fuses local and global feature representations, enabling comprehensive data modeling.
• The proposed DIN model is evaluated on three benchmark HAR datasets: PAMAP2, WISDM, and OPPORTUNITY.

Main Results:

• The DIN model consistently outperformed state-of-the-art baselines across all tested datasets.
• Achieved high F1-scores: 92.05% on PAMAP2, 98.17% on WISDM, and 91.55% on OPPORTUNITY.
• Demonstrated efficiency with fewer parameters and FLOPs, validated by real-time execution on a Raspberry Pi P3 system.

Conclusions:

• The Dual-branch Interactive Network (DIN) effectively integrates CNN and Transformer merits for superior performance in sensor-based HAR.
• The proposed model offers a compelling solution for real-time HAR applications on lightweight wearable devices.
• The DIN presents a significant advancement for ubiquitous HAR computing scenarios, balancing accuracy and computational efficiency.