Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD.

Area of Science:

• Computational linguistics
• Natural language processing
• Cognitive science

Background:

• Previous work established semantic vectors from word-word co-occurrence statistics.
• Pointwise mutual information with small windows and cosine distance yielded optimal representations for psychological tasks.

Purpose of the Study:

• To investigate the impact of stop-lists, word stemming, and singular value decomposition (SVD) on semantic representations.
• To explore the benefits of using a significantly larger text corpus.
• To introduce a new semantic task for evaluation.

Main Methods:

• Systematic computational analysis of word-word co-occurrence statistics.
• Application of stop-lists and word stemming techniques.
• Dimensionality reduction via singular value decomposition (SVD).
• Evaluation across multiple psychologically relevant semantic tasks, including a new one and a standard TOEFL task.

Main Results:

• Improved SVD-based methods were developed for generating semantic representations.
• State-of-the-art performance was achieved on a standard TOEFL semantic task.
• The study identified potential pitfalls and misleading outcomes from incomplete systematic analyses.

Conclusions:

• Further refinement of semantic vector extraction is possible through techniques like SVD and larger corpora.
• Systematic investigation is crucial to avoid misleading results in semantic representation studies.
• The developed methods offer enhanced capabilities for understanding semantic relationships in text.