VillageNet: Graph-based, Easily-interpretable, Unsupervised Clustering for Broad Biomedical Applications.

Area of Science:

• Data Science
• Machine Learning
• Computational Statistics

Background:

• Clustering large, high-dimensional datasets is crucial for uncovering latent information.
• Existing methods often require prior knowledge of the number of clusters, limiting their applicability.

Purpose of the Study:

• To develop an unsupervised clustering algorithm, Village-Net, capable of handling large, high-dimensional data.
• To enable autonomous determination of the optimal number of clusters.
• To provide an efficient and effective solution for complex data analysis.

Main Methods:

• Village-Net employs a two-phase approach: K-Means clustering to form initial 'villages' (subsets).
• A weighted network is constructed where nodes represent villages and edges represent proximity.
• Community detection using Walk-likelihood Community Finder (WLCF) is applied to the network for optimal clustering.

Main Results:

• Village-Net demonstrates competitive performance on real-world datasets, outperforming state-of-the-art methods.
• The algorithm excels in Normalized Mutual Information (NMI) scores.
• Its computational efficiency is highlighted with a time complexity of O(N*k*d).

Conclusions:

• Village-Net is an effective unsupervised algorithm for clustering large, high-dimensional datasets.
• It autonomously determines the optimal number of clusters, offering flexibility.
• The algorithm's efficiency and performance make it suitable for large-scale data analysis.