n-Gram based language processing using Twitter dataset to identify COVID-19 patients.

Area of Science:

• Natural Language Processing
• Machine Learning
• Computational Social Science

Background:

• The proliferation of multilingual electronic documents necessitates efficient language categorization and processing techniques.
• Accurate language identification is crucial for analyzing social media data, such as tweets, for public health monitoring (e.g., COVID-19).
• Character and word n-gram based methods offer efficient document categorization, but memory efficiency remains a challenge.

Purpose of the Study:

• To propose a novel framework for language categorization and processing of tweets using n-gram techniques and machine learning.
• To evaluate the framework's performance in terms of accuracy, precision, recall, and F-measure across different n-gram lengths.
• To develop a scalable solution for analyzing multilingual social media data for health monitoring and sustainable society goals.

Main Methods:

• Implementation of an n-gram based language categorization technique for tweets.
• Integration of the Linear Support Vector Machine (LSVM) for further tweet processing.
• Evaluation of the framework's performance metrics (accuracy, precision, recall, F-measure) based on n-gram length.

Main Results:

• The proposed framework demonstrates effective language categorization and processing capabilities for multilingual tweets.
• Performance evaluation provides insights into optimal n-gram lengths for specific metrics.
• The framework is designed for scalability, accommodating large volumes of social media data.

Conclusions:

• The developed framework offers an efficient and scalable solution for multilingual language processing of social media content.
• This approach supports applications in public health surveillance, such as identifying COVID-19 positive patients from tweets.
• The framework contributes to advancements in natural language processing and supports the goal of a sustainable society through data-driven insights.