STADe: Sensory Temporal Action Detection via Temporal-Spectral Representation Learning.

Area of Science:

• Computer Vision
• Internet of Things
• Sensor Data Analysis

Background:

• Temporal Action Detection (TAD) traditionally focuses on video data.
• Extending TAD to sensor data presents challenges: varying sampling rates, complex patterns, and noise.
• Existing TAD models struggle with the unique characteristics of sensory signals.

Purpose of the Study:

• To propose a novel model, Sensory Temporal Action Detection (STADe), for effective action detection in sensor data.
• To address the limitations of current TAD approaches when applied to diverse sensory signals.
• To facilitate future research in sensor-based temporal action detection.

Main Methods:

• STADe utilizes Fourier kernels and adaptive frequency filtering to capture temporal and frequency features.
• Employs deep fusion at multiple resolutions and scales for adaptability to diverse data.
• Introduces a cross-cascade predictor for bidirectional and temporal dependencies within action categories.

Main Results:

• STADe demonstrates superior performance compared to state-of-the-art TAD models on sensory data.
• Experiments were conducted on one public and three newly established diverse sensor datasets.
• The model effectively handles varying sampling rates and action durations inherent in sensory signals.

Conclusions:

• The proposed STADe model offers a robust solution for temporal action detection in sensor data.
• STADe's adaptive mechanisms make it versatile for various sensor types and data characteristics.
• The established datasets will serve as a valuable resource for advancing research in sensory TAD.