The Impact of Sparse Datasets When Harmonizing Data from Studies with Different Measures of the Same Construct

Area of Science:

• Prevention Science
• Psychometrics
• Biostatistics

Background:

• Integrative data analysis (IDA) is increasingly used in prevention science to pool individual participant data from multiple studies.
• A common challenge in IDA is the presence of sparse datasets due to the use of different measures for the same construct across studies.
• Measurement bias can arise from data sparseness, potentially affecting the validity of IDA findings.

Purpose of the Study:

• To introduce a graph theory method for characterizing patterns of data sparseness in IDA.
• To investigate, using simulations, how different sparseness patterns impact measurement bias within various measurement models.
• To provide guidance on assessing and mitigating bias in IDA due to data sparseness.

Main Methods:

• A graph theory approach was developed to summarize patterns of data sparseness.
• Simulations were conducted on 1000 datasets with varying sparseness levels across three measurement models (single common factor, hierarchical, bifactor).
• Bayesian methods were employed to estimate model parameters and evaluate measurement bias.

Main Results:

• Measurement bias resulting from data sparseness is contingent upon the strength of the general factor, the specific measurement model utilized, and the degree of indirect linkage among measures.
• The study demonstrated how different sparseness patterns can differentially impact parameter estimates and bias.
• An illustrative example using a synthesis dataset on youth depression from 16 prevention trials was presented.

Conclusions:

• The pattern of data sparseness significantly influences measurement bias in integrative data analysis.
• Investigators should consider the unique sparseness patterns within their synthesis datasets.
• Simulation methods are recommended for proactively exploring and understanding potential bias in IDA studies.