Effect Modification Analyses in Individual Participant Data Meta-Analyses: A Systematic Review

How often effect modification analyses in individual participant data meta-analyses (IPDMAs) are prespecified and consistently reported in protocols and corresponding reports remains unclear.

To investigate the planning of effect modification analyses in IPDMA protocols and evaluate their concordance with corresponding published reports.

In this systematic review, Medline, Embase, the Cochrane Database of Systematic Reviews, and PROSPERO were searched for protocols of IPDMAs of randomized clinical trials from database inception until December 31, 2021, and their corresponding published study reports until January 31, 2024. Paired reviewers independently identified eligible protocols and reports and extracted data. Planning and reporting characteristics of effect modification analyses were summarized, and matched protocol-report pairs were compared for concordance in prespecification of effect modification analyses.

Of 356 protocols, 336 (94.4%) had planned effect modification analyses (median, 6 [IQR, 4-10] per protocol). Of those, 12 (3.6%) stated an anticipated direction, 176 (52.4%) planned interaction tests, and 37 (11.0%) planned within-trial analyses. Of 265 protocols using continuous variables as effect modifiers, 15 (5.7%) planned to treat all continuous variables as continuous and 72 (27.2%) planned to categorize all variables without providing justification thresholds. Of 166 identified reports, 149 (89.8%) reported effect modification analyses (median, 7 [IQR, 3-9] per study report). Among those 149 reports, 84 (56.4%) reported prespecified effect modification analyses, 125 (83.9%) reported interaction tests, and 3 (2.0%) stated anticipated effect directions. Among 133 reports using continuous variables, 87 (65.4%) used specified thresholds without providing justification for chosen cut points. Among 166 report-protocol pairs, 32 (19.3%) showed agreement on the number and factors of effect modification analyses. Of 124 mismatched pairs, 95 (76.6%) omitted 1 or more planned effect modification analyses in reports (74 [59.7%] without explanation), and 74 (59.7%) added unplanned effect modification analyses. Among 156 comparable pairs, 15 reports (9.6%) claimed prespecification for effect modification analyses absent from protocols and 84 reports (53.8%) failed to claim prespecification for effect modification analyses prespecified in protocols.

In this systematic review of IPDMAs, protocols and reports often failed to prespecify anticipated effect modification directions, distinguish within-trial from between-trial analyses, and appropriately handle continuous variables. Concordance between protocols and reports was poor, with frequent omission of planned and addition of unplanned effect modification analyses.