Detection and evaluation of clusters within sequential data

Area of Science:

• Data Science
• Computational Biology
• Bioinformatics

Background:

• Sequential data is prevalent in various fields, presenting challenges due to high dimensionality, sparsity, and noise.
• Extracting meaningful insights from complex sequential processes requires robust methods to handle data dependencies.

Purpose of the Study:

• To evaluate novel clustering algorithms, derived from Block Markov Chains theory, on real-world sequential data.
• To determine if these algorithms can effectively generate useful low-dimensional representations from sparse, high-dimensional sequences.

Main Methods:

• Application of new clustering algorithms designed for sequential data.
• Empirical study across diverse real-world datasets including animal movement (GPS), DNA sequences, text, and financial data.
• Analysis of the extracted low-dimensional representations for their ability to encode sequential structure and reveal underlying process characteristics.

Main Results:

• The algorithms successfully extracted low-dimensional representations from diverse real-world sequential data.
• These representations effectively captured the inherent sequential structure within the datasets.
• The identified representations provided novel insights into the complex processes under study.

Conclusions:

• The Block Markov Chain-based clustering algorithms are effective for extracting meaningful low-dimensional representations from complex, real-world sequential data.
• This approach offers a promising method for gaining deeper understanding in fields dealing with sequential information.
• The study validates the utility of these algorithms beyond synthetic data, demonstrating their applicability in practical scenarios.