Simultaneous Clustering and Model Selection: Algorithm, Theory and Applications.

Area of Science:

• Data Science
• Machine Learning
• Computational Statistics

Background:

• Clustering algorithms are widely studied, but model selection remains a significant challenge.
• Existing methods often treat clustering and model selection separately, limiting performance.
• Difficulty in selecting the optimal number of clusters and their assignments hinders practical applications.

Purpose of the Study:

• To jointly address clustering and model selection problems.
• To develop a robust method for recovering an ideal affinity tensor from imperfect input data.
• To improve the accuracy and efficiency of clustering analysis.

Main Methods:

• Recovering an ideal affinity tensor by enforcing low-rank, sparsity, and rank-1 sum constraints.
• Developing a joint optimization framework for model selection and clustering.
• Proposing an alternative formulation for efficient online stochastic optimization for large-scale problems.

Main Results:

• Significantly improved affinity input by repairing corrupted entries and handling outliers.
• Directly determined the number of clusters and their membership from the recovered tensor.
• Demonstrated superior performance across various applications and settings.

Conclusions:

• The proposed joint approach effectively solves clustering and model selection simultaneously.
• The method enhances data quality and provides robust cluster identification.
• The algorithm is scalable and adaptable to diverse real-world scenarios.