Fragile preferences: A deep dive into order effects in large language models

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Decision Science

Background:

• Large language models (LLMs) are used in high-stakes decision support.
• Existing research notes position biases but lacks systematic analysis.
• Underlying preference structures linked to biases are not well understood.

Purpose of the Study:

• To comprehensively study position biases in LLMs across different domains.
• To investigate the link between biases and underlying preference structures.
• To identify novel biases and propose mitigation strategies.

Main Methods:

• Systematic analysis of position biases across multiple LLMs.
• Experiments conducted in resume comparison and color selection domains.
• Application of an extended rational choice framework to classify preferences.

Main Results:

• Consistent order effects observed, including a quality-dependent shift.
• A novel 'name bias' was identified, favoring certain names irrespective of demographics.
• Order effects were shown to cause selection of strictly inferior options.

Conclusions:

• LLMs exhibit distinct failure modes, including order and name biases.
• These biases can distort judgments and lead to suboptimal choices.
• Mitigation strategies, including temperature parameter adjustment, can help recover true preferences.