Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

Area of Science:

• Natural Language Processing
• Artificial Intelligence Ethics

Background:

• Advancements in language models raise concerns about the misuse of automatically generated text.
• Detecting machine-generated content is crucial for maintaining trust in online information and human interaction.
• Current detection methods often require computationally expensive language models.

Purpose of the Study:

• To propose a simple, feature-based classifier for detecting machine-generated text.
• To offer an accessible and cost-effective alternative to existing detection methods.
• To analyze the impact of different sampling methods on the characteristics of generated text.

Main Methods:

• Development of a feature-based classifier using intrinsic differences between human and machine text.
• Evaluation of the classifier's performance against established, more resource-intensive methods.
• Experimentation with various sampling methods to identify resulting text flaws.

Main Results:

• The proposed feature-based classifier achieves competitive performance compared to expensive language model-based approaches.
• The detection method provides an accessible "first line-of-defense" against the abuse of language models.
• Experimental findings indicate that distinct sampling methods result in unique flaws within generated text.

Conclusions:

• A simple feature-based classifier can effectively detect machine-generated text, rivaling more complex methods.
• This approach offers an accessible solution for combating the misuse of AI-generated content.
• Understanding sampling method-induced flaws is key to improving both generation and detection techniques.