Optimal architecture for a sentiment analysis transformer with multihead attention and genetic crossover

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Sentiment analysis is crucial for strategic surveillance, online reputation management, and customer satisfaction.
• Improving the accuracy and adaptability of sentiment analysis models remains a significant challenge.
• Current methods often struggle with complex textual data and generalization.

Purpose of the Study:

• To present a novel method for enhancing sentiment analysis accuracy and efficiency.
• To combine Transformer architecture with evolutionary optimization techniques.
• To develop a lighter, more efficient, and robust sentiment analysis model.

Main Methods:

• Utilized pre-trained models for rich textual representations (embeddings and language features).
• Developed OAST-MAGC (Optimal Architecture for a Sentiment Analysis Transformer with Multihead Attention and Genetic Crossover).
• Integrated dynamic pruning of attention heads and genetic crossover for final layer weights optimization.

Main Results:

• Achieved a high accuracy of 95.96% and an F1-score of 96%.
• Demonstrated superior performance and robustness compared to existing sentiment analysis techniques.
• The dynamic pruning and genetic crossover significantly improved convergence speed and model efficiency.

Conclusions:

• The proposed OAST-MAGC method offers significant improvements in sentiment analysis accuracy and efficiency.
• The integration of dynamic pruning and genetic crossover leads to more generalized and less overfitted models.
• This approach opens new possibilities for processing complex textual data in various applications.