Simple-random-sampling-based multiclass text classification algorithm.

Area of Science:

• Computer Science
• Artificial Intelligence
• Natural Language Processing

Background:

• Multiclass text classification (MTC) is crucial for many applications but faces challenges with space-time overhead in big data environments.
• Existing MTC algorithms often struggle with efficiency, necessitating novel approaches for large-scale datasets.

Purpose of the Study:

• To address the space-time overhead issue in multiclass text classification for big data.
• To propose and evaluate a new MTC algorithm based on simple random sampling and text retrieval.

Main Methods:

• Investigated token frequency distribution in a Chinese web document collection to reexamine the power law.
• Developed a simple-random-sampling-based MTC (SRSMTC) algorithm utilizing token-level memory and a text retrieval approach.
• Validated the algorithm's performance on the TanCorp dataset.

Main Results:

• The SRSMTC algorithm demonstrated state-of-the-art performance in multiclass text classification.
• Achieved significantly reduced space-time requirements compared to existing methods.
• The token-level memory and text retrieval approach proved effective for efficient classification.

Conclusions:

• The SRSMTC algorithm offers an efficient and effective solution for multiclass text classification in big data scenarios.
• The findings suggest that sampling-based methods combined with text retrieval can overcome the limitations of traditional MTC algorithms.
• This research contributes a practical approach for handling large-scale text data classification tasks.