Particle swarm optimization-based NLP methods for optimizing automatic document classification and retrieval

Area of Science:

• Natural Language Processing
• Machine Learning
• Deep Learning

Background:

• Text classification is crucial for NLP tasks like sentiment analysis and information retrieval.
• Existing models face challenges with multi-class classification and complex documents.

Purpose of the Study:

• To introduce the PBX model, a hybrid approach combining deep learning and traditional machine learning for improved text classification.
• To enhance model performance through BERT pre-training, ConvXGB classification, and Particle Swarm Optimization (PSO) for hyperparameter tuning.

Main Methods:

• Utilized BERT for deep learning-based text pre-training.
• Employed the ConvXGB module for text classification.
• Applied Particle Swarm Optimization (PSO) for hyperparameter optimization.
• Evaluated the model on diverse datasets: 20 Newsgroups, Reuters-21578, and AG News.

Main Results:

• The PBX model demonstrated superior performance over existing methods in accuracy, precision, recall, and F1 score.
• Achieved 95.0% accuracy and 94.9% F1 score on the AG News dataset.
• Ablation studies confirmed the significant contributions of PSO, BERT, and ConvXGB.

Conclusions:

• The PBX model offers a robust solution for challenging text classification tasks.
• Future research will address performance on smaller categories and broaden application scope.