Analysis of Web Spam for Non-English Content: Toward More Effective Language-Based Classifiers.

Area of Science:

• Computer Science
• Natural Language Processing
• Information Retrieval

Background:

• Web spammers continuously evolve tactics to deceive search engines and achieve higher rankings.
• Existing search engine anti-spam mechanisms, like Google Penguin, face challenges in detecting sophisticated spam.
• The effectiveness of spam detection features can be influenced by the language of the web page.

Purpose of the Study:

• To investigate the impact of page language on web spam detection features.
• To evaluate the efficacy of Google Penguin against Arabic web spam.
• To develop and validate a language-based web spam classifier for improved detection accuracy.

Main Methods:

• Exploration of language-specific spam detection features.
• Case study analysis of Arabic spam pages and comparison with English counterparts.
• Development and evaluation of a manual-labeled corpus of Arabic web pages (benign and spam).
• Creation of a browser plug-in for real-time spam detection and filtering.

Main Results:

• Google Penguin demonstrates limited effectiveness against a significant portion of Arabic spam pages.
• The optimal set of spam detection features varies considerably based on page language.
• Language-based web spam classifiers significantly outperform Google Penguin in capturing spam content.

Conclusions:

• Web spam detection strategies must account for language-specific characteristics.
• A language-aware approach is crucial for developing robust and effective web spam classifiers.
• The developed Arabic web spam classifier and browser plug-in offer practical solutions for users and researchers.