Scalable Cluster Tendency Assessment for Streaming Activity Data using Recurring Shapelets.

Area of Science:

• Data Science
• Machine Learning
• Pattern Recognition

Background:

• Interpreting cluster structure in high-velocity data streams is crucial for real-time event detection.
• Human activities exhibit repeating patterns, necessitating methods that capture these temporal shapes.
• Existing Visual Assessment of Cluster Tendency (VAT) models struggle with scalability for streaming data or fail to recognize pattern shapes.

Purpose of the Study:

• To develop an incremental algorithm for interpreting cluster evolution in streaming activity data.
• To address the limitations of existing VAT algorithms in handling large data streams and shape pattern detection.
• To enable timely detection of interesting events and anomalous movements in activity data.

Main Methods:

• Proposed an incremental algorithm, a novel relative of the Visual Assessment of Cluster Tendency (VAT) model.
• Developed a Dictionary-of-Shapes (DoS) to identify and store repeating patterns (shapelets) from data chunks.
• Implemented an intelligent Maximin Random Sampling (MMRS) scheme for scalable VAT image creation and incremental updates.

Main Results:

• The proposed method successfully visualizes clusters in long data streams.
• The algorithm efficiently identifies repeating patterns and anomalous movements in activity data.
• Experimental results on upper limb activity datasets validate the method's effectiveness and scalability.

Conclusions:

• The novel incremental VAT algorithm effectively interprets cluster evolution in streaming data.
• The Dictionary-of-Shapes and MMRS scheme enable scalable and accurate analysis of shape patterns.
• This approach enhances the detection of anomalies and understanding of complex activity patterns in real-time.