Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization.

Area of Science:

• Computational statistics
• Data mining
• Machine learning

Background:

• Convex clustering offers theoretical advantages but faces adoption barriers due to computational intensity and poor visualization.
• Existing methods for convex clustering are computationally expensive and limit the granularity of solutions.

Purpose of the Study:

• To introduce Algorithmic Regularization for efficient convex clustering path estimation.
• To develop the Convex Clustering via Algorithmic Regularization Paths (CARP) algorithm.
• To enhance the visualization capabilities of convex clustering solutions.

Main Methods:

• Algorithmic Regularization: An iterative one-step approximation scheme for estimating regularization paths.
• Global convergence guarantee for the approximate path under specific assumptions.
• CARP algorithm implementation for computing the full clustering solution path.

Main Results:

• CARP achieves over a 100-fold speed-up compared to existing convex clustering methods.
• The algorithm provides a finer approximation grid, enabling more detailed analysis.
• Enhanced visualization through convex clustering-based dendrograms and dynamic path-wise plots.

Conclusions:

• Algorithmic Regularization effectively addresses the computational and visualization limitations of convex clustering.
• CARP offers a significant performance improvement and richer visualization options for complex datasets.
• The open-source R package clustRviz facilitates the application of these advanced convex clustering techniques.