Adaptive Boosting LLMs for Text Classification

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Large-scale language models (LLMs) show advanced capabilities across various NLP tasks.
• The increasing capabilities of LLMs create uncertainty in the future of text categorization research.
• The effectiveness of LLMs specifically for text classification remains an open question.

Purpose of the Study:

• To investigate the extent to which text classification has advanced using LLMs.
• To introduce a novel framework, Recurrent Generative Pre-trained Transformer (RGPT), for dedicated text classification LLMs.

Main Methods:

• RGPT is an adaptive boosting framework that creates a sequence of base learners.
• It dynamically modulates training data distribution and iteratively fine-tunes LLMs.
• Base learners are progressively integrated using historical prediction trajectories for specialization.

Main Results:

• RGPT demonstrated superior performance compared to eight state-of-the-art pre-trained language models.
• It outperformed seven cutting-edge LLMs across four benchmark datasets.
• An average performance gain of 2.90% was achieved.

Conclusions:

• RGPT represents a significant advancement in specialized LLMs for text classification.
• The proposed framework effectively leverages LLM potential for improved text categorization accuracy.
• RGPT offers a promising direction for future research in specialized language modeling.