Large language models (LLMs) as agents for augmented democracy

Area of Science:

• Computational Social Science
• Political Science
• Artificial Intelligence

Background:

• Traditional methods for gauging public opinion may not capture nuanced policy preferences.
• Understanding citizen alignment with government programs is crucial for democratic governance.

Purpose of the Study:

• To explore an augmented democracy system using large language models (LLMs) to enhance data on citizen policy preferences.
• To assess the accuracy of LLMs in predicting individual and aggregate political choices based on government programs.
• To investigate if LLM-augmented data can capture policy preferences beyond party affiliations.

Main Methods:

• Fine-tuning off-the-shelf large language models (LLMs) on citizen preferences related to Brazilian presidential election policies.
• Employing a train-test cross-validation setup to evaluate LLM predictive accuracy at individual and aggregate levels.
• Comparing LLM predictions against a 'bundle rule' and non-augmented probabilistic samples.

Main Results:

• LLMs demonstrated higher accuracy in predicting out-of-sample individual preferences compared to the 'bundle rule'.
• LLM-augmented probabilistic samples provided more accurate estimates of aggregate population preferences than non-augmented samples.
• LLM-augmented data successfully captured policy preferences that extended beyond simple party alignment.

Conclusions:

• LLM-augmented data augmentation is a promising method for capturing nuanced citizen policy preferences.
• This approach offers a more accurate representation of public opinion, potentially improving digital governance and participatory city initiatives.
• The study highlights the potential of AI in enhancing democratic processes by better understanding citizen alignment with policy proposals.