A novel transformer-based semantic feature extraction method for multi-label text classification

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Multi-label text classification is crucial for assigning multiple categories to documents.
• Existing methods struggle with complex label dependencies and fine-grained feature extraction.

Purpose of the Study:

• To propose a novel Transformer-based semantic feature extraction method (TMSFE) for multi-label text classification.
• To improve the accuracy and efficiency of feature extraction in multi-label classification tasks.

Main Methods:

• Developed TMSFE integrating label-specific query embeddings and multi-head attention.
• Employed a DeBERTaV3-based Transformer encoder for joint document and label semantic modeling.
• Utilized a SimCSE-Based latent semantic space module to enhance feature extraction efficiency.

Main Results:

• TMSFE demonstrated superior performance compared to baseline models.
• Achieved lower Hamming loss, indicating improved classification accuracy.
• Showcased higher feature extraction accuracy.

Conclusions:

• The proposed TMSFE method effectively addresses challenges in multi-label text classification.
• TMSFE offers a promising approach for enhanced semantic feature extraction and classification accuracy.