SensorNet: An Adaptive Attention Convolutional Neural Network for Sensor Feature Learning.

Area of Science:

• Machine Learning
• Deep Learning
• Sensor Data Analysis

Background:

• Pretrained neural networks often exhibit poor portability to new applications with limited sensor data.
• Developing generalizable sensor feature learning models is crucial for diverse real-world applications.

Purpose of the Study:

• To develop a generalizable neural network, SensorNet, for effective sensor feature learning across various applications.
• To address the challenge of poor portability of existing models to new domains with scarce data.

Main Methods:

• Integration of self-attention flexibility with the multi-scale feature locality of convolution.
• Introduction of patch-wise self-attention with stacked multi-heads for enriched sensor feature representation.
• Development of a significantly smaller model (0.83 M parameters) compared to state-of-the-art hybrid baselines (3.87 M parameters).

Main Results:

• SensorNet achieves state-of-the-art performance on the SHL'18 activity recognition dataset.
• Pretrained SensorNet fine-tuned on smaller datasets shows significant improvements (up to 5% on WISDM) and achieves top accuracy on EEG datasets (SLEEP-EDF-20).
• Demonstrates superior performance and generalizability compared to top models in activity recognition and other sensor-based tasks.

Conclusions:

• SensorNet offers a highly generalizable and efficient solution for sensor feature learning.
• The model's architecture enhances feature representation and enables effective transfer learning.
• SensorNet demonstrates strong potential for pervasive applications requiring robust sensor data analysis with limited data.