A Pattern Dictionary Method for Anomaly Detection.

Area of Science:

• Data Science
• Machine Learning
• Signal Processing

Background:

• Anomaly detection is crucial for identifying unusual patterns in time series and sequence data.
• Existing methods may struggle with complex patterns and require robust baselines for accurate detection.

Purpose of the Study:

• To propose a compression-based anomaly detection method using a pattern dictionary.
• To develop a robust system for identifying anomalous patterns in sequential data.
• To establish a framework for creating health baselines for anomaly detection.

Main Methods:

• Utilizing a pattern dictionary to learn complex patterns in training data.
• Employing sequence complexity measures (parsed phrases, codelength) as anomaly scores.
• Combining the pattern dictionary with universal source coders for atypicality detection.
• Deriving a non-asymptotic upper bound for LZ78 parser using the Lambert W function.

Main Results:

• The pattern dictionary method effectively detects anomalies by assessing sequence complexity.
• Combining with universal source coders creates a powerful atypicality detector.
• A novel non-asymptotic bound for LZ78 was derived, defining the anomaly score range.
• The framework was illustrated for establishing health baselines against deviations.

Conclusions:

• The proposed pattern dictionary method offers a powerful and flexible approach to anomaly detection in sequential data.
• The method provides a quantitative anomaly score and can be enhanced with universal source coders.
• The derived theoretical bound contributes to understanding the method's performance limits.