Spot the bot: the inverse problems of NLP.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Distinguishing human-written from bot-generated text is crucial in the digital age.
• Previous methods often focused on specific bot types, limiting generalizability.
• A need exists for versatile features applicable across various bot-generated texts.

Purpose of the Study:

• To develop a method for distinguishing any human-written text from any bot-generated text.
• To identify efficient and versatile features for text classification.
• To evaluate the performance of simple classifiers using these features.

Main Methods:

• Analysis of large-scale, coarse-grained structure of the language semantic space.
• Dataset construction separating bots, not just their texts, for robust testing.
• Feature derivation using clustering (Wishart, K-Means, fuzzy variations) and entropy-complexity measures.
• Classification using simple models like Support Vector Machine, Decision Tree, and Random Forest.

Main Results:

• Achieved over 96% classification quality in large-scale simulations.
• Demonstrated the effectiveness of derived features with simple classifiers.
• Observed varying performance across different language families.

Conclusions:

• The proposed method effectively distinguishes human-written from bot-generated texts.
• Versatile features derived from semantic space analysis are key to high classification accuracy.
• The approach offers a robust solution for detecting AI-generated content across diverse linguistic contexts.