Ideological bias in the production of research findings

When studying policy-relevant topics, researchers' policy preferences may shape analytical decisions and results interpretations. Detecting this bias is challenging because the research process is not normally part of an observed experimental setting. Our study exploits an opportunity to observe 158 researchers working independently in 71 teams during an experiment. After being asked their position on immigration policy, they used the same data to answer the same empirical question: Does immigration affect public support for social welfare programs? The researchers estimated 1253 alternative regression models, and the estimated impacts ranged from strongly negative to strongly positive. We find that teams composed of pro-immigration researchers estimated more positive impacts of immigration on public support for social programs, while anti-immigration teams estimated more negative impacts. The differences arise because different teams adopted different model specifications. The underlying research design decisions are the mechanism through which ideology enters the process of producing parameter estimates.