Estimation, Forecasting, and Anomaly Detection for Nonstationary Streams Using Adaptive Estimation.

Area of Science:

• Data Science
• Machine Learning
• Statistical Modeling

Background:

• Streaming data presents computational and statistical challenges, including memory/speed constraints and concept drift.
• Handling temporal structures like trend and periodicity in streaming data remains difficult.

Purpose of the Study:

• To propose a penalized-regression framework, the real-time adaptive component (RAC), for efficient streaming data analysis.
• To address concept drift and computational limitations inherent in streaming data environments.

Main Methods:

• The RAC framework utilizes adaptive filtering techniques for estimation.
• It employs a specified basis for local structure and a LASSO-like penalty to prevent overfitting.
• The procedure is enhanced with a streaming anomaly detection capability.

Main Results:

• Simulated data experiments indicate RAC is a competitive tool across various scenarios.
• Real cyber-security data analysis demonstrates the method's practical promise.

Conclusions:

• The RAC procedure offers a robust solution for streaming data analysis, adeptly managing concept drift and computational demands.
• Its integration of anomaly detection further enhances its utility in real-world applications like cyber-security.