A conditional random field based approach for high-accuracy part-of-speech tagging using language-independent...

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Machine Learning

Background:

• Part-of-speech (POS) tagging is crucial for understanding text grammar and is vital for numerous Natural Language Processing (NLP) tasks.
• Urdu text processing presents unique challenges, including morphological richness, lack of capitalization, and spelling variations, necessitating specialized automatic tagging systems.
• Existing Urdu POS tagging research often requires complex feature engineering, highlighting the need for more efficient and effective methods.

Purpose of the Study:

• To develop and evaluate a supervised machine learning model for automatic Urdu part-of-speech (POS) tagging.
• To address the challenges of Urdu text processing by utilizing language-independent features.
• To achieve state-of-the-art performance in Urdu POS tagging using a Conditional Random Field (CRF) model.

Main Methods:

• A Conditional Random Field (CRF) based supervised classifier was developed for 33 Urdu POS categories.
• The model utilized language-independent features extracted from the Urdu news dataset MM-POST (119,276 tokens).
• A simpler strategy was employed, focusing on word-level features within context windows and word length.

Main Results:

• The proposed CRF model achieved an overall classification accuracy of 96.1% on the MM-POST dataset.
• The approach demonstrated superiority over previous Urdu POS tagging research.
• The model's effectiveness was attributed to the efficient utilization of selected word-level features and context.

Conclusions:

• The developed CRF model offers a superior and simpler strategy for Urdu POS tagging compared to existing methods.
• The study highlights the effectiveness of language-independent features and a focused feature set for accurate Urdu POS tagging.
• This research contributes a high-performing, state-of-the-art automatic POS tagging system for Urdu, advancing NLP capabilities for the language.