Attributing authorship via the perplexity of authorial language models

Area of Science:

• Computational Linguistics
• Digital Forensics
• Stylometry

Background:

• Authorship attribution traditionally relies on quantitative stylometric methods.
• The emergence of Large Language Models (LLMs) presents novel opportunities for authorship analysis.
• Existing methods may not fully leverage the capabilities of modern AI in text analysis.

Purpose of the Study:

• To develop and evaluate a novel authorship attribution technique utilizing fine-tuned LLMs.
• To compare the performance of the proposed LLM-based method against current state-of-the-art approaches.
• To investigate the linguistic features that contribute to successful authorship attribution.

Main Methods:

• Further pretraining of LLMs on individual author writing samples to create Authorial Language Models (ALMs).
• Assigning a questioned document to the author whose ALM yields the lowest perplexity (highest predictability).
• Analyzing word-level predictability to inspect linguistic patterns driving attribution decisions.

Main Results:

• The proposed LLM-based authorship attribution method meets or exceeds state-of-the-art performance on standard datasets.
• The approach allows for direct inspection of linguistic patterns influencing attribution.
• Content words were found to carry more authorship information than function words, challenging prior assumptions.

Conclusions:

• Fine-tuned LLMs offer a powerful and effective approach to authorship attribution.
• The method provides interpretable insights into attribution drivers.
• The findings suggest a re-evaluation of the role of different word classes in stylometry.