Leveraging Contextual Sentences for Text Classification by Using a Neural Attention Model.

Area of Science:

• Computer Science
• Artificial Intelligence
• Natural Language Processing

Background:

• Deep learning models for text classification often struggle to effectively utilize contextual information.
• Traditional feature extraction methods may not fully capture the nuances of text data.

Purpose of the Study:

• To explore methods for incorporating context information into deep learning frameworks for text classification.
• To propose and evaluate novel deep learning algorithms that leverage context for improved performance.

Main Methods:

• Developed two classification algorithms: a convolutional neural network (CNN) and a bidirectional long short-term memory (BiLSTM) network, both enhanced with context.
• Designed attention mechanisms at both sentence and word levels to integrate context into feature representations.
• Extracted additional features using traditional methods to augment representation.

Main Results:

• The proposed CNN and BiLSTM models demonstrated superior time efficiency and accuracy compared to baseline models with fundamental attention (AM) architectures.
• Integration of context information at multiple levels (word and sentence) increased feature representation diversity.
• Experimental validation on two datasets confirmed the advantages of the context-aware models.

Conclusions:

• Incorporating context information through advanced attention mechanisms significantly improves deep learning-based text classification.
• The proposed CNN and BiLSTM models offer a more efficient and accurate approach to text classification tasks.
• Attention structures at word and sentence levels are effective in enriching feature diversity for better classification outcomes.