Text normalization for named entity recognition in Vietnamese tweets.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Machine Learning

Background:

• Named Entity Recognition (NER) identifies and categorizes entities like persons, locations, and organizations in text.
• Twitter data presents unique challenges for NER due to its noisy, brief, and irregular nature.
• Existing NER approaches are limited for Vietnamese tweets, highlighting a research gap.

Purpose of the Study:

• To develop and evaluate a robust NER system specifically for Vietnamese tweets.
• To address the challenges posed by the informal and noisy characteristics of social media text.

Main Methods:

• A novel tweet normalization technique is proposed to correct spelling errors using Dice's coefficient or n-grams.
• A Support Vector Machine (SVM) learning algorithm is utilized for classification.
• The model incorporates six distinct feature types for enhanced recognition accuracy.

Main Results:

• The proposed method was trained on over 40,000 named entities.
• Evaluation on a test set of 3,186 named entities demonstrated strong performance.
• The system achieved a state-of-the-art F1 score of 82.13% for Vietnamese tweet NER.

Conclusions:

• The developed method effectively handles the complexities of NER in Vietnamese tweets.
• Tweet normalization is a crucial preprocessing step for improving NER performance.
• The system sets a new benchmark for Vietnamese tweet Named Entity Recognition.