Understanding the success and failure of online political debate: Experimental evidence using large language models

Online political debate is frequently lamented for being toxic, partisan, and counterproductive. However, we know little about how core elements of political debate (justification, tone, willingness to compromise, and partisanship) affect its quality. Using text-based treatments experimentally manipulated with a large language model, we test how these elements causally affect the quality of open-text responses about issues important to the US and UK public. We find substantial evidence that differences in justification, tone, and willingness to compromise, but not partisanship, affect the quality of subsequent discourse. Combined, these elements increase the probability of high-quality responses by roughly 1.6 to 2 times and substantially increase openness to alternative viewpoints. Despite the ability to bring about substantial changes in discourse quality, we find no evidence of changes in political attitudes themselves. Our findings demonstrate how adapting approaches to online debate can foster healthy democratic interactions but have less influence on changing minds.