Researcher allegiance in research on psychosocial interventions: meta-research study protocol and pilot study

One potential source of bias in randomised clinical trials of psychological interventions is researcher allegiance (RA). The operationalisation of RA differs strongly across studies, and there is not a generally accepted method of operationalising or measuring it. Furthermore, it remains unclear as to how RA affects the outcomes of trials and if it results in better outcomes for a preferred intervention. The aim of this project is to develop and validate a scale that accurately identifies RA, contribute to the understanding of the impact that RA has in a research setting and to make recommendations for addressing RA in practice.

A scale will first be developed and validated to measure RA in psychotherapy trials. The scale will be validated by surveying authors of psychotherapy trials to assess their opinions, beliefs and preferences of psychotherapy interventions. Furthermore, the scale will be validated for use outside the field of psychotherapy. The validated checklist will then be used to examine two potential mechanisms of how RA may affect outcomes of interventions: publication bias (by assessing grants) and risk of bias (RoB). Finally, recommendations will be developed, and a feasibility study will be conducted at a national mental health agency in The Netherlands. Main analyses comprise inter-rater reliability of checklist items, correlations to examine the relationship between checklist items and author survey (convergent validity) as well as checklist items and trial outcomes and multivariate meta-regression techniques to assess potential mechanisms of how allegiance affects trial outcomes (publication bias and RoB).

This study has been reviewed and approved by the Scientific and Ethical Review Board (VCWE) at the Vrije Universiteit Amsterdam. Study result and advancements will also be published on the Open Science Framework. Furthermore, main findings will be disseminated through articles in international peer-reviewed open access journals. Results and recommendations will be communicated to the Cochrane Collaboration, the Campbell Collaboration and other funding agencies.