Enhanced industrial text classification via hyper variational graph-guided global context integration.

Area of Science:

• Natural Language Processing
• Machine Learning
• Data Science

Background:

• Pre-trained models excel at local context processing for text classification.
• Industrial text datasets, especially with small sample sizes, present unique classification challenges.
• Existing methods often struggle to capture the global context crucial for industrial text data.

Purpose of the Study:

• To propose an approach for industrial text classification using enhanced global context representation from pre-trained models.
• To develop a method that effectively handles small sample sizes in industrial text datasets.
• To improve the accuracy and F1 scores in industrial text classification tasks.

Main Methods:

• Leveraging BERT pre-trained models to extract primary text representations and local context embeddings.
• Constructing a text information entropy matrix fused with BERT embeddings and a hyper variational graph through iterative updates.
• Utilizing capsule networks for purification and expansion of BERT text features, followed by fusion with the hypergraph representation.

Main Results:

• The proposed model achieved 86.82% accuracy and 82.87% F1 score on the CHIP-CTC dataset.
• On the CLUEEmotion2020 dataset, accuracy reached 61.22% and F1 score was 51.56%.
• The model demonstrated strong performance on an industrial patent dataset with 91.84% accuracy and 79.71% F1 score, outperforming baselines across all tested datasets.

Conclusions:

• The developed method effectively addresses the classification problem for industrial domain text, particularly with limited data.
• The integration of global context representation significantly enhances classification performance.
• The approach offers a robust solution for diverse industrial text classification applications.