A flexible framework for anomaly Detection via dimensionality reduction.

Area of Science:

• Data Science
• Machine Learning
• Statistics

Background:

• Anomaly detection is crucial but challenging for large, high-dimensional datasets.
• Existing methods often struggle with scalability and performance in high dimensions.

Purpose of the Study:

• To present a general anomaly detection framework and its Python implementation, DRAMA.
• To evaluate DRAMA's robustness and competitiveness on diverse datasets.

Main Methods:

• Dimensionality reduction and unsupervised clustering to identify data prototypes.
• Anomaly detection based on distances from prototypes in latent or original spaces.
• Implementation as a flexible Python package (DRAMA) with various options.

Main Results:

• DRAMA demonstrates robustness and high competitiveness, especially in datasets up to 3000 dimensions.
• The framework is effective on both simulated and real-world datasets.
• Anomalies are identified by their significant distance from identified data prototypes.

Conclusions:

• DRAMA offers a flexible and effective solution for anomaly detection in high-dimensional data.
• Its adaptability makes it suitable for online learning, active learning, and imbalanced datasets.
• The package also facilitates outlier clustering for further analysis.