Hierarchical gated recurrent neural network with adversarial and virtual adversarial training on text classification

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Document classification is crucial for content management and understanding.
• Hierarchical Attention Networks (HAN) are effective for ambiguous document classification by processing documents at word and sentence levels.
• Standard HAN models can suffer from overfitting due to redundant training parameters.

Purpose of the Study:

• To propose a variant of the Hierarchical Attention Network (HAN) that mitigates overfitting.
• To enhance document classification accuracy by incorporating adversarial and virtual adversarial perturbations.
• To validate the effectiveness of the proposed method on multiple datasets.

Main Methods:

• A novel Hierarchical Attention Network (HAN) variant was developed.
• Adversarial and virtual adversarial perturbations were applied to word and sentence representations.
• The proposed model was evaluated on eight publicly available document classification datasets.

Main Results:

• The proposed HAN variant with perturbations outperformed standard HAN models.
• The model achieved state-of-the-art performance on several benchmark datasets.
• Perturbations were shown to effectively alleviate overfitting and boost performance.

Conclusions:

• Adversarial perturbations are an effective strategy to improve Hierarchical Attention Networks (HAN) for document classification.
• The proposed HAN variant offers a robust solution to overfitting in text classification tasks.
• This research contributes to advancing the state-of-the-art in document classification accuracy and robustness.