Time-Aware and Power-Law Retention Gating Mechanisms in LSTMs for Irregularly Sampled Sensor Data: A Survey

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Sensor data from IoT devices and wearables are often irregularly sampled, creating temporal gaps.
• Standard Long Short-Term Memory (LSTM) networks assume equidistant data and lack explicit time dependence, leading to biases like exponential memory decay.

Purpose of the Study:

• To address the limitations of standard LSTM networks in handling irregularly sampled sensor data.
• To survey and taxonomize existing strategies for improving temporal awareness in sequence models.
• To propose a unified design framework for temporal forget gates in sensor data analysis.

Main Methods:

• Survey and taxonomize time-aware mechanisms and power-law retention schemes for LSTM networks.
• Conduct comparative analysis of these strategies.
• Articulate a unified design framework for temporal forget gates.

Main Results:

• Both time-aware and power-law forgetting strategies struggle to jointly adapt memory attenuation rate and decay profile to temporal context and sensor input.
• Existing methods often employ functionally redundant mechanisms.
• A unified design framework is proposed, defining essential properties for temporal forget gates.

Conclusions:

• Future sequence models for sensing applications must integrate temporal-gap awareness with adaptive retention dynamics.
• This integration is crucial for robust learning from irregular sensor streams and capturing complex temporal dependencies.