Sparse Count Data Clustering Using an Exponential Approximation to Generalized Dirichlet Multinomial Distributions.

Area of Science:

• Computational statistics
• Machine learning
• Data mining

Background:

• Clustering high-dimensional, sparse count data is computationally challenging.
• Generalized Dirichlet multinomial (GDM) distributions offer accuracy but suffer from slow parameter estimation.
• Exponential-family approximations provide efficient training without dimensionality reduction.

Purpose of the Study:

• To develop an efficient exponential-family approximation to GDM distributions, termed EGDM.
• To create a novel clustering algorithm for count data using an EGDM mixture model.
• To introduce a method for determining the optimal number of EGDM components using the Minimum Message Length (MML) criterion.

Main Methods:

• Derivation of an exponential-family approximation to GDM distributions (EGDM).
• Development of a mixture model based on EGDM.
• Parameter learning using the deterministic annealing expectation-maximization (DAEM) approach.
• Optimal component selection via the Minimum Message Length (MML) criterion.

Main Results:

• The proposed EGDM mixture model demonstrates superior clustering performance on text, image, and video data.
• The EGDM approach significantly reduces computation time compared to standard GDM methods.
• Empirical experiments validate the effectiveness and efficiency of the EGDM clustering algorithm.

Conclusions:

• The EGDM model offers an efficient and accurate solution for clustering high-dimensional count data.
• The DAEM algorithm effectively learns parameters for the EGDM mixture model.
• The MML criterion provides a reliable method for selecting the optimal number of clusters.