Enhancing the Generalization for Text Classification through Fusion of Backward Features

Area of Science:

• Deep Learning
• Natural Language Processing
• Machine Learning

Background:

• Deep learning models often fail to generalize well, particularly with sentences containing conflicting sentiment phrases.
• Existing models focus on phrase-level features, neglecting overall semantic understanding, leading to unstable classifications.
• Attention mechanisms in current models can be overly influenced by specific phrases, hindering accurate sentiment analysis.

Purpose of the Study:

• To develop a method that improves the generalization ability of deep learning models in sentiment analysis and sarcasm detection.
• To address the instability of models when classifying sentences with mixed positive and negative sentiment.
• To redirect model attention away from contradictory sentiment words for more robust semantic understanding.

Main Methods:

• Proposed a novel two-stream network architecture incorporating an auxiliary network.
• Utilized a gradient reversal layer within the auxiliary network to reverse feature gradients during training.
• Integrated a feature projection layer to merge backward features with normal features in the main network.

Main Results:

• The proposed method demonstrated improved performance on sentiment analysis datasets, with a 0.5% accuracy increase.
• Significant improvements were observed in sarcasm detection datasets, achieving a 2.1% increase in accuracy.
• The approach effectively enhanced model stability when dealing with sentences containing opposite sentiment phrases.

Conclusions:

• The developed method successfully mitigates the issue of attention being overly focused on specific phrases.
• By scattering attention away from conflicting sentiment cues, the model achieves a more holistic semantic interpretation.
• This technique offers a promising direction for enhancing the generalization and robustness of deep learning models in NLP tasks.