The influence of preprocessing on text classification using a bag-of-words representation

Area of Science:

• Natural Language Processing
• Machine Learning

Background:

• Text classification (TC) is vital for organizing information.
• Preprocessing is a common step in TC applications.
• The impact of various preprocessing combinations on TC performance is not fully understood.

Purpose of the Study:

• To systematically investigate the effect of different preprocessing method combinations on text classification accuracy.
• To identify optimal preprocessing strategies for benchmark text corpora.

Main Methods:

• Conducted extensive text classification experiments using four benchmark corpora.
• Evaluated all possible combinations of five to six basic preprocessing methods.
• Utilized three machine learning methods with standard training and testing protocols.
• Employed a bag-of-words (BOW) representation for text data.

Main Results:

• At least one preprocessing combination significantly improved TC accuracy across all tested datasets.
• Stopword removal was the most effective single method for three datasets.
• For one dataset, lowercasing was the only effective single method, with spelling correction and lowercasing yielding the best results.
• Minimal improvements were observed with HTML tag removal, spelling correction, punctuation removal, and character reduction in some cases.

Conclusions:

• Systematic exploration of preprocessing method combinations is highly recommended for improving text classification.
• The optimal preprocessing strategy varies depending on the dataset.
• Preprocessing significantly enhances TC accuracy, especially when using BOW representations.