Position-context additive transformer-based model for classifying text data on social media.

Area of Science:

• Natural Language Processing
• Machine Learning
• Social Media Analytics

Background:

• The proliferation of social media text necessitates advanced classification models.
• Transformer-based models are effective for natural language processing tasks.
• Existing models may not fully capture positional and contextual information in social media text.

Purpose of the Study:

• To introduce a novel Position-Context Additive transformer model (PCA model) for improved social media text classification.
• To enhance text representation by integrating word position and context.
• To refine the additive attention mechanism within transformer architectures.

Main Methods:

• Developed a two-phase approach: Phase I integrates improved word embeddings (position) with Bi-LSTM (context).
• Phase II enhances transformer models by improving the additive attention mechanism.
• Evaluated the PCA model on six datasets for health-related social media text classification.

Main Results:

• The PCA model achieved improved F1-Scores (0.2-10.2%) on five datasets compared to state-of-the-art results.
• Outperformed three other transformer-based models in four datasets, with F1-score improvements of 0.1-2.1%.
• Demonstrated a positive correlation between training data volume and performance accuracy.

Conclusions:

• The PCA model offers superior performance for social media text classification.
• Integrating positional and contextual information is crucial for enhancing text representation.
• Larger training datasets positively impact the accuracy of transformer-based text classification models.