Continual learning with attentive recurrent neural networks for temporal data classification.

Area of Science:

• Deep Learning
• Artificial Intelligence
• Machine Learning

Background:

• Continual learning aims to develop models that learn sequentially without forgetting past knowledge.
• Temporal-based continual learning techniques are underutilized despite their potential for time-series data.
• Catastrophic forgetting remains a significant challenge in continual learning, especially with temporal data.

Purpose of the Study:

• To address the underutilization of temporal-based continual learning methods.
• To propose a novel solution for catastrophic forgetting in temporal data learning.
• To introduce Temporal Teacher Distillation (TTD) for task-incremental learning on temporal data.

Main Methods:

• Developed Temporal Teacher Distillation (TTD), a novel method based on attentive recurrent neural networks.
• TTD addresses catastrophic forgetting by considering the Rotation Hypothesis, Redundant Hypothesis, and Recover Hypothesis.
• Evaluated TTD on the WIreless Sensor Data Mining (WISDM) and Split-QuickDraw-100 datasets in a task-incremental setting.

Main Results:

• TTD significantly outperforms state-of-the-art methods in temporal-based continual learning.
• Achieved up to 14.6% improvement in accuracy.
• Demonstrated up to 45.1% improvement in forgetting measures.

Conclusions:

• TTD effectively mitigates catastrophic forgetting in temporal data classification.
• The proposed method offers a significant advancement for continual learning with temporal data.
• This research pioneers continual learning for real-world incremental temporal data classification using attentive recurrent neural networks.