Simulating Public Opinion: Comparing Distributional and Individual-Level Predictions from LLMs and Random Forests

Area of Science:

• Computational Social Science
• Artificial Intelligence
• Political Science

Background:

• Traditional agent-based models use simplified rules for human decision-making.
• Modeling information flow is key to understanding polarization and opinion dynamics.

Purpose of the Study:

• Explore Large Language Models (LLMs) as high-fidelity agents for simulating opinions.
• Assess LLMs' ability to predict individual responses using real survey data.
• Compare LLM simulations to traditional models in a zero-shot setting.

Main Methods:

• Conditioned LLMs on 2020 American National Election Studies (ANES) survey data.
• Used Jensen-Shannon distance and F1-score for evaluation.
• Compared LLM simulations against a supervised Random Forest model.

Main Results:

• LLMs showed comparable performance to Random Forest at the individual level.
• LLMs consistently produced aggregate opinion distributions closer to empirical data.
• LLMs demonstrated strong predictive accuracy in a zero-shot setting.

Conclusions:

• LLMs show promise for simulating complex opinion dynamics.
• LLMs offer a novel approach to modeling belief systems in computational social science.
• LLMs can capture nuanced, context-sensitive human decision-making in simulations.