A divided and prioritized experience replay approach for streaming regression

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Streaming learning involves agents learning from continuous data streams.
• Catastrophic forgetting, where models lose previously learned information, is a significant challenge in this setting.
• Existing methods for mitigating catastrophic forgetting are varied, but improvements are still sought.

Purpose of the Study:

• To present a novel divided and prioritized experience replay approach for streaming regression.
• To address the issue of catastrophic forgetting in online learning.
• To improve the model's ability to retain and utilize knowledge from important, rare events.

Main Methods:

• A divided and prioritized experience replay strategy is proposed.
• The prediction space is divided to manage data streams effectively.
• Experience replay prioritizes poorly estimated observations for focused training.
• A real-world dataset was used for comparison against the standard sliding window approach.

Main Results:

• The proposed method demonstrates improved performance on rare, important events compared to the sliding window approach.
• Statistical power analysis confirms the enhancement in handling infrequent but significant data points.
• A trade-off in performance for more common observations was noted.
• Rephrasing the problem as binary classification highlighted improvements on rare events.

Conclusions:

• The divided and prioritized experience replay approach effectively mitigates catastrophic forgetting in streaming regression.
• The method offers a valuable perspective on improving learning from rare events.
• This technique provides a robust solution for handling imbalanced data in streaming learning scenarios.