Automated Phrase Mining from Massive Text Corpora.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Data Mining

Background:

• Phrase mining is crucial for text analysis, with applications in information retrieval and topic modeling.
• Current methods often require complex linguistic analyzers and human expert input, limiting their adaptability and automation.
• Existing state-of-the-art models, even data-driven ones, are not fully automated and struggle with new domains and languages without costly adaptation.

Purpose of the Study:

• To introduce AutoPhrase, a novel framework for fully automated phrase mining.
• To develop a language-agnostic approach that leverages general knowledge bases.
• To improve the effectiveness and efficiency of phrase extraction compared to existing methods.

Main Methods:

• Proposed a novel framework, AutoPhrase, for automated phrase mining.
• Designed the system to support any language with an available general knowledge base (e.g., Wikipedia).
• The framework can optionally utilize a Part-of-Speech (POS) tagger for enhanced performance.

Main Results:

• AutoPhrase demonstrated significant improvements in both effectiveness and efficiency.
• Evaluated on five real-world datasets spanning different domains and languages.
• Achieved superior performance compared to state-of-the-art phrase mining techniques.

Conclusions:

• AutoPhrase offers a highly effective and efficient solution for automated phrase mining.
• The framework's language-agnostic design and reliance on knowledge bases enhance its applicability.
• AutoPhrase can be extended to effectively identify single-word quality phrases.