Human-interpretable clustering of short text using large language models.

Area of Science:

• Natural Language Processing
• Machine Learning
• Data Mining

Background:

• Clustering short text is challenging due to low word co-occurrence.
• Traditional methods like doc2vec and latent Dirichlet allocation have limitations in capturing semantic meaning.

Purpose of the Study:

• To demonstrate the efficacy of large language models (LLMs) in clustering short texts.
• To compare LLM-based clustering with traditional methods.
• To explore LLMs for cluster validation.

Main Methods:

• Generating text embeddings using large language models (LLMs).
• Applying Gaussian mixture modeling to cluster embeddings.
• Comparing LLM-generated clusters with doc2vec and latent Dirichlet allocation outputs.
• Quantifying cluster quality using human reviewers and a generative LLM.

Main Results:

• LLM-based embeddings capture semantic nuances, overcoming limitations of traditional methods.
• Clusters generated using LLMs and Gaussian mixture modeling are more distinctive and human-interpretable.
• A generative LLM shows strong agreement with human reviewers in cluster validation.
• Comparison reveals biases in both LLM and human coding, questioning human coding as the sole validation standard.

Conclusions:

• Large language models offer a powerful solution for short text clustering.
• LLMs can bridge the validation gap in clustering by providing reliable interpretation.
• The study challenges the conventional reliance on human coding for cluster validation, highlighting LLM potential.