Large-scale moral machine experiment on large language models.

Area of Science:

• Artificial Intelligence
• Ethics in Technology
• Autonomous Systems

Background:

• Large Language Models (LLMs) are rapidly advancing, raising questions about their ethical decision-making.
• Integrating LLMs into autonomous driving systems requires assessing their alignment with human moral values.
• Previous research examined a limited number of LLMs, necessitating a broader analysis.

Purpose of the Study:

• To evaluate the moral judgments of 52 diverse LLMs in autonomous driving scenarios.
• To assess the alignment of LLM ethical decisions with human preferences.
• To investigate the impact of model size, updates, and architecture on moral alignment.

Main Methods:

• Conjoint analysis framework applied to assess LLM responses in ethical dilemmas.
• Evaluation of proprietary (GPT, Claude, Gemini) and open-source (Llama, Gemma) LLMs.
• Analysis of factors including model size (parameters), version updates, and architecture.

Main Results:

• Proprietary and larger open-source LLMs (>10B parameters) showed closer alignment with human moral judgments.
• A negative correlation was observed between model size and deviation from human judgments in open-source models.
• Model updates did not consistently enhance ethical alignment; some LLMs overemphasized specific ethical principles.

Conclusions:

• Increasing LLM size may naturally improve human-like moral judgments.
• Implementing LLMs in autonomous driving requires balancing ethical alignment with computational efficiency.
• Cultural context is crucial for AI moral decision-making in autonomous systems.