Legal Decision Biases in GPT: A Comparison with Human Judgment.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Legal Technology

Background:

• Human legal decision-making is susceptible to procedural biases, such as evidence order and intermediate evaluations.
• Legal professionals, including judges, demonstrate these biases in criminal case assessments.
• The increasing use of large language models (LLMs) in law necessitates understanding their susceptibility to similar biases.

Purpose of the Study:

• To investigate whether advanced LLMs (GPT-4o, GPT-5.2) exhibit procedural biases in legal judgment tasks.
• To compare LLM biases with human judgment patterns from prior research.
• To assess the effectiveness of prompt engineering in mitigating observed AI biases.

Main Methods:

• A controlled legal judgment task was adapted from human studies, manipulating evidence order and the requirement for intermediate guilt judgments.
• GPT-4o and GPT-5.2 were tested on simplified criminal cases with systematic procedural variations.
• Model responses were compared to human judgments, and prompt engineering strategies were evaluated for bias reduction.

Main Results:

• GPT-4o displayed significant order effects and a distinct form of evaluation bias.
• GPT-5.2 exhibited similar, though less pronounced, procedural sensitivities.
• Prompt engineering strategies had a limited and inconsistent effect on reducing biases in both models.

Conclusions:

• Advanced LLMs, including GPT-4o and GPT-5.2, are vulnerable to normatively irrelevant procedural influences in legal judgment tasks.
• Current prompt engineering techniques are insufficient to reliably eliminate these biases.
• Caution is advised when deploying LLMs as decision-support systems in high-stakes legal contexts due to their susceptibility to bias.