Global considerations in hierarchical clustering reveal meaningful patterns in data

Area of Science:

• Data science
• Bioinformatics
• Machine learning

Background:

• Hierarchical clustering organizes data using tree-like structures.
• Bottom-up (BU) agglomerative algorithms are commonly used by default in unsupervised machine learning.
• Existing methods may not fully capture complex data relationships.

Purpose of the Study:

• To evaluate the effectiveness of global hierarchical clustering algorithms (top-down and glocal) compared to traditional bottom-up methods.
• To introduce and assess a novel density-based top-down hierarchical clustering algorithm.
• To demonstrate the utility of hierarchical clustering for improving data annotation and identifying errors.

Main Methods:

• Comparison of top-down (TD), glocal, and bottom-up (BU) hierarchical clustering algorithms.
• Evaluation of algorithm performance using statistical criteria based on expert-labeled data.
• Application of algorithms to diverse datasets, including gene expression, stock data, and protein families.
• Development and testing of a novel TD algorithm based on genuine data point density.

Main Results:

• Top-down (TD) and glocal algorithms are better suited for revealing meaningful patterns than BU algorithms.
• TD algorithms excel at identifying global patterns, while BU algorithms are advantageous for finer data granularity.
• The novel density-based TD algorithm outperformed existing divisive and agglomerative methods.
• Application to protein sequences identified overlooked functional annotations.

Conclusions:

• Global approaches like TD and glocal algorithms should be considered in exploratory clustering.
• Unsupervised clustering methods can enhance manually created data mappings, such as protein families.
• Hierarchical clustering provides insights into erroneous and missed annotations, improving data quality.