Time-lag reporting bias in interrupted time series studies: a methodological study

Time-lag bias occurs when the speed of publication depends on the nature of the results (eg, whether they support the authors' hypothesis). Time-lag bias has not been examined among interrupted time series (ITS) studies.

To investigate whether the time-to-publication of ITS studies with a protocol is influenced by their results.

We searched eight bibliographic databases and 14 other sources for published protocols of ITS studies, up to 31 December 2022. We then searched for reports of results that matched these protocols by their primary ITS research question. The time to publication for each ITS study was calculated as the time between publication date of the report of results and that of the protocol or the end date of data collection period, whichever was later. In each report, all eligible results pertaining to the primary research question were classified as favorable or not favorable to the interruption, based on statistical significance and direction of effect. We estimated the median time-to-publication from a Kaplan-Meier curve of the rate of publication of results over time. To investigate time-lag reporting bias, we conducted multivariable linear regression to examine whether the percentage of favorable results per study was associated with time-to-publication.

We identified 120 ITS study protocols published between 2009 and 2022, of which 44/120 (37%) had at least 1 matched report of results, with a corresponding 860 results. Among these 44 protocols, the median time-to-publication was 2.0 years (95% confidence interval 1.6-2.6) from the protocol's submission. The median percentage of results favorable to the interruption per study was 20% (interquartile range 5%-50%). For each percentage point increase in the percentage of results favorable to the interruption per study, the time-to-publication increased by 3 days (95% confidence interval 5 days earlier to 10 days later; P = .505).

We identified that very few protocols for ITS are published. In those published, there was uncertainty about whether studies reporting results that are favorable to the interruption are published sooner. Publication of protocols for ITS studies needs to become a normative practice.

ITS studies are a particular type of design used to evaluate the effects of interventions delivered to populations (such as a national multimedia campaign to reduce smoking rates). It has been found that studies showing positive findings may get published faster than those showing neutral or negative findings, a problem called "time-lag bias." If positive results are published earlier, clinicians and policy makers may make decisions based on incomplete and optimistic evidence. However, there has been no examination of whether time-lag bias occurs in ITS studies. In this study, we searched for protocols of ITS studies, which are documents that describe the planned methods for undertaking a study. We then searched the published literature to see whether the results from study had been published. We analyzed the data to determine how long it took for studies to be published and whether studies reporting positive results were published faster than those with negative or neutral results. Out of thousands of ITS studies conducted between 2000 and 2020, we found only 120 published ITS protocols. Approximately one-third of these protocols had their results published, with publication taking an average of 2 years. There was no clear evidence that studies with positive results were published faster than those with negative or neutral results, although our estimates were uncertain. Because few researchers publish protocols before starting an ITS study, it remains difficult to detect time-lag bias. Health researchers should routinely publish protocols prior to conducting the study.