Informing Harmonization Decisions in Integrative Data Analysis: Exploring the Measurement Multiverse.

Area of Science:

• Psychometrics
• Data Science
• Behavioral Science

Background:

• Integrative data analysis (IDA) combines multiple datasets, requiring careful data harmonization.
• Decisions in logical and analytic harmonization can cumulatively impact study outcomes.
• The influence of these harmonization decisions on psychometric models is not well understood.

Purpose of the Study:

• To investigate the cumulative effects of harmonization decisions in IDA.
• To assess the impact of varying harmonization strategies on psychometric model parameters and factor scores.
• To examine the robustness of the relationship between alcohol use and delinquency estimates under different harmonization approaches.

Main Methods:

• Conducted an IDA using three datasets (N=2245) on alcohol use and delinquency.
• Employed moderated nonlinear factor analysis (MNLFA) for analytic harmonization and factor score generation.
• Systematically varied logical and analytic harmonization decisions 72 times to assess cumulative influence.

Main Results:

• MNLFA parameter estimates showed variability across different harmonization paths.
• Estimates for factor scores and regression parameters linking delinquency to alcohol use were less affected by harmonization choices.
• Subtle differences in harmonization decisions had a notable impact on measurement model parameters.

Conclusions:

• Factor scores derived from MNLFA appear relatively robust to variations in data harmonization decisions.
• Measurement model parameters are more sensitive to the specific choices made during data harmonization.
• Researchers should be mindful of the cumulative impact of harmonization decisions in IDA studies.