Robust Detection of Watermarks for Large Language Models Under Human Edits

Area of Science:

• Artificial Intelligence
• Computer Science
• Information Security

Background:

• Watermarking is crucial for differentiating AI-generated text from human writing.
• Human edits significantly degrade existing watermark detection accuracy.
• Current methods struggle with the noise introduced by text modifications.

Purpose of the Study:

• Introduce a novel method, the Truncated Goodness-of-Fit (Tr-GoF) test, for robust watermark detection.
• Address the challenge of detecting watermarked text under substantial human edits.
• Improve the reliability of identifying AI-generated content.

Main Methods:

• Modeled human edits using mixture model detection.
• Developed the Tr-GoF test based on a truncated goodness-of-fit approach.
• Analyzed the test's asymptotic optimality and detection efficiency.

Main Results:

• Proved Tr-GoF achieves optimal robust detection for Gumbel-max watermarks under significant edits.
• Demonstrated Tr-GoF's adaptive nature, not requiring precise edit levels or LLM specifications.
• Showed Tr-GoF attains high detection efficiency in moderate modification regimes.
• Highlighted the inadequacy of sum-based detection rules against edit-induced noise.

Conclusions:

• The Tr-GoF test offers superior robustness and efficiency in detecting watermarked text with human edits.
• Tr-GoF outperforms existing sum-based methods, especially under significant text modifications.
• Empirical results validate Tr-GoF's effectiveness on various LLMs like OPT and LLaMA.