Document vectorization method using network information of words.

Area of Science:

• Natural Language Processing
• Information Retrieval
• Computational Linguistics

Background:

• Traditional document vectorization relies heavily on word frequency metrics like term frequency (TF) and TF-inverse document frequency (TF-IDF).
• These frequency-based methods may not fully capture the semantic nuances and unique characteristics of a document.
• Understanding word relationships within a document is crucial for richer representation.

Purpose of the Study:

• To propose and evaluate a novel document vectorization method based on word relational characteristics.
• To compare the effectiveness of this new method against established frequency-based vectorization techniques.
• To determine if relational information provides a more accurate representation for document clustering.

Main Methods:

• Document vectorization using two types of word relational information: centrality measures and co-occurrence counts.
• Utilizing cosine similarity for clustering analysis on vectors derived from relational information.
• Comparing results with clustering analysis using traditional frequency-based vectors and vectors of tie weights between words.

Main Results:

• Clustering analysis using relational information vectors showed comparable or superior accuracy to frequency-based methods.
• The most accurate clustering results were achieved using vectors of tie weights between words.
• While results are based on a small dataset, they indicate the potential of relational characteristics for improved document representation.

Conclusions:

• Document vectorization using word relational characteristics can yield more accurate results than frequency-based methods.
• Centrality measures and co-occurrence patterns offer valuable insights into document content.
• Further research on larger datasets is warranted to generalize these findings.