Quantifying the prevalence and impact of overreaching causal claims in social science

Across the social sciences, many studies use cross-sectional designs that reveal associations but are generally unable to support direct causal claims, yet authors of such articles may make or imply causal claims anyway. Here, to examine the prevalence of such 'overreaching' causal language, we analysed 194,631 cross-sectional articles using large language models. Over the period 1980-2024, an average of 46% of articles contained causal language in their titles or abstracts, where the annual rate has risen almost threefold since 2000 from 20% to 60%. To examine the effects of such language, we conducted a human-subjects experiment (N = 1, 105), finding that readers frequently indicate abstracts with this phrasing provide causal evidence but that methodological labels (β = -0.4, 95% confidence interval -0.56 to -0.19) and associational wording (β = -0.3, 95% confidence interval -0.43 to -0.07) reduce this tendency. Experiments with five LLMs revealed that model summaries of these articles (N = 100 each) can amplify causal overstatement, removing hedges and introducing causal claims where articles used strictly associational phrasing; however, prompting caution diminishes this pattern.