Experimental study on short-text clustering using transformer-based semantic similarity measure.

Area of Science:

• Natural Language Processing
• Machine Learning
• Information Retrieval

Background:

• Sentence clustering is vital for text processing and semantic similarity measurement.
• Traditional methods struggle with semantically similar sentences lacking common words.
• Low-dimensional continuous representations are underutilized in evaluating clustering performance.

Purpose of the Study:

• To present a novel sentence similarity measure using embedding representations.
• To evaluate its impact on partitional, hierarchical, and fuzzy clustering methods.
• To compare the effectiveness of different pre-training models for this similarity measure.

Main Methods:

• Implemented an embedding-based sentence similarity measure.
• Applied the measure to evaluate three clustering techniques (partitional, hierarchical, fuzzy).
• Trained the similarity measure on two state-of-the-art pre-training models.
• Utilized hierarchical clustering for a text summarization task.

Main Results:

• The embedding-based similarity measure significantly improved text clustering performance.
• Hierarchical clustering, using the new measure, demonstrated superior results.
• The approach also led to enhanced performance in text summarization.

Conclusions:

• Embedding-based sentence similarity measures offer superior semantic understanding for text clustering.
• The proposed implementation effectively enhances both text clustering and summarization tasks.
• This method provides a robust framework for evaluating and improving NLP tasks reliant on semantic similarity.