A systematic review of the quality of reporting of simulation studies about methods for the analysis of complex...

This study describes the characteristics and quality of reporting for published computer simulation studies about statistical methods to analyze complex longitudinal (i.e., repeated measures) patient-reported outcomes (PROs); we included methods for longitudinal latent variable measurement and growth models and response shift.

Scopus, PsycINFO, PubMed, EMBASE, and Social Science Citation Index were searched for English-language studies published between 1999 and 2016 using selected keywords. Extracted information included characteristics of the study purpose/objectives, simulation design, software, execution, performance, and results. The quality of reporting was evaluated using published best-practice guidelines.

A total of 1470 articles were reviewed and 42 articles met the inclusion criteria. The majority of the included studies (73.8%) investigated an existing statistical method, primarily a latent variable model (95.2%). Most studies specified the population model, including variable distributions, mean parameters, and correlation/covariances. The number of time points and sample size(s) were reported by all studies, but justification for the selected values was rarely provided. The majority of the studies (52.4%) did not report on model non-convergence. Bias, accuracy, and model fit were commonly reported performance metrics. All studies reported results descriptively, and 26.2% also used an inferential method.

While methodological research on statistical analyses of complex longitudinal PRO data is informed by computer simulation studies, current reporting practices of these studies have not been consistent with best-practice guidelines. Comprehensive reporting of simulation methods and results ensures that the strengths and limitations of the investigated statistical methods are thoroughly explored.